2020-11-16T19:53:45.015 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-16T19:53:45.017 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631141 --owner scheduled_kchai@teuthology --block --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} --verbose --name kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi --lock -- /tmp/teuthology-worker.3v33mnm6.tmp 2020-11-16T19:53:45.061 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T19:53:45.129 INFO:teuthology.run:Config: arch: x86_64 archive_path: /home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631141 branch: wip-kefu-testing-2020-11-16-1845 description: rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} email: tchaikov@gmail.com first_in_suite: false job_id: '5631141' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-kefu-testing-2020-11-16-1845 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 osd_async_recovery_min_cost: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - Health detail - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - Health detail sha1: 911cc35794d871071a8bebfb0919482263b1d18b ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 911cc35794d871071a8bebfb0919482263b1d18b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-kefu-testing-2020-11-16-1845 sha1: 911cc35794d871071a8bebfb0919482263b1d18b owner: scheduled_kchai@teuthology priority: 90 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 911cc35794d871071a8bebfb0919482263b1d18b sleep_before_teardown: 0 suite: rados suite_branch: wip-kefu-testing-2020-11-16-1845 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-11-16-1845/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 911cc35794d871071a8bebfb0919482263b1d18b tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: crush-failure-domain: osd k: 2 m: 1 name: isaprofile plugin: isa technique: reed_sol_van min_size: 2 objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 4000 write_append_excl: false teuthology_branch: master timestamp: 2020-11-16_12:11:03 tube: smithi user: kchai verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.7053 2020-11-16T19:53:45.130 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-11-16-1845/qa; will attempt to use it 2020-11-16T19:53:45.131 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-11-16-1845/qa/tasks 2020-11-16T19:53:45.131 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-16T19:53:45.133 INFO:teuthology.task.internal:Checking packages... 2020-11-16T19:53:45.154 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '911cc35794d871071a8bebfb0919482263b1d18b' 2020-11-16T19:53:45.155 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-16T19:53:45.155 INFO:teuthology.packaging:ref: None 2020-11-16T19:53:45.156 INFO:teuthology.packaging:tag: None 2020-11-16T19:53:45.156 INFO:teuthology.packaging:branch: wip-kefu-testing-2020-11-16-1845 2020-11-16T19:53:45.156 INFO:teuthology.packaging:sha1: 911cc35794d871071a8bebfb0919482263b1d18b 2020-11-16T19:53:45.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-kefu-testing-2020-11-16-1845 2020-11-16T19:53:45.331 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-7219-g911cc357-1bionic 2020-11-16T19:53:45.331 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-16T19:53:45.341 INFO:teuthology.task.internal:no buildpackages task found 2020-11-16T19:53:45.342 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-16T19:53:45.367 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-16T19:53:45.368 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T19:53:45.417 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:53:55.435 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:54:05.450 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:54:15.467 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:54:25.481 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:54:35.498 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:54:45.517 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:54:55.534 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:55:05.552 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:55:15.571 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:55:25.588 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:55:35.605 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:55:45.623 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:55:55.638 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:56:05.655 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:56:15.672 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:56:25.688 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:56:35.709 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:56:45.723 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:56:55.739 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:57:05.761 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:57:15.772 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:57:25.784 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:57:35.803 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:57:45.820 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:57:55.839 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:58:05.857 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:58:15.874 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-16T19:58:25.890 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_kchai@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631141', 'arch': 'x86_64'} 2020-11-16T19:58:25.909 DEBUG:teuthology.lock.ops:locked ubuntu@smithi092.front.sepia.ceph.com, ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T19:58:25.960 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5a71513bf8>, signals=[15]) 2020-11-16T19:58:26.173 INFO:teuthology.provision.fog.smithi013:Scheduling deploy of ubuntu 18.04 2020-11-16T19:58:26.178 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of ubuntu 18.04 2020-11-16T19:58:26.440 INFO:teuthology.orchestra.console:Power off smithi013 2020-11-16T19:58:26.441 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T19:58:26.465 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:58:26.480 INFO:teuthology.orchestra.console:Power off smithi092 2020-11-16T19:58:26.481 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-16T19:58:26.501 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:58:30.492 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:58:46.183 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:58:50.208 INFO:teuthology.orchestra.console:Power off for smithi013 completed 2020-11-16T19:58:50.306 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2020-11-16T19:58:50.310 INFO:teuthology.orchestra.console:Power on smithi013 2020-11-16T19:58:50.311 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T19:58:50.334 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:58:50.408 INFO:teuthology.orchestra.console:Power on smithi092 2020-11-16T19:58:50.408 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-16T19:58:50.432 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:58:54.362 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:58:54.456 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:58:54.575 INFO:teuthology.orchestra.console:Power on for smithi013 completed 2020-11-16T19:58:54.676 INFO:teuthology.provision.fog.smithi013:Waiting for deploy to finish 2020-11-16T19:58:58.571 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-16T19:59:02.715 INFO:teuthology.orchestra.console:Power on for smithi092 completed 2020-11-16T19:59:02.816 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish 2020-11-16T20:01:40.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:03.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:04.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:13.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:28.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:37.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:46.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:51.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:51.450 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-16T20:02:55.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:57.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:02:57.465 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-16T20:03:03.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:03:03.481 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-16T20:03:09.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:03:09.494 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-16T20:03:15.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:03:15.577 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T20:03:15.579 INFO:teuthology.orchestra.run.smithi013:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T20:03:15.967 INFO:teuthology.provision.fog.smithi013:Node is ready 2020-11-16T20:03:15.967 INFO:teuthology.orchestra.run.smithi013:> hostname 2020-11-16T20:03:16.015 INFO:teuthology.orchestra.run.smithi013.stdout:smithi013.front.sepia.ceph.com 2020-11-16T20:03:16.015 INFO:teuthology.orchestra.run.smithi013:> grep smithi013.front.sepia.ceph.com /etc/hosts 2020-11-16T20:03:16.063 INFO:teuthology.orchestra.run.smithi013.stdout:172.21.15.13 smithi013.front.sepia.ceph.com smithi013 2020-11-16T20:03:16.064 INFO:teuthology.orchestra.run.smithi013:> sudo hostname smithi013 2020-11-16T20:03:16.190 INFO:teuthology.orchestra.run.smithi013:> sudo sed -i -e 's/smithi013.front.sepia.ceph.com/smithi013/g' /etc/hosts 2020-11-16T20:03:16.206 INFO:teuthology.orchestra.run.smithi013:> sudo sed -i -e 's/172.21.15.13/172.21.15.13/g' /etc/hosts 2020-11-16T20:03:16.264 INFO:teuthology.orchestra.run.smithi013:> sudo sed -i -e 's/smithi013.front.sepia.ceph.com/smithi013/g' /etc/hostname 2020-11-16T20:03:16.320 INFO:teuthology.orchestra.run.smithi013:> sudo hostname smithi013 2020-11-16T20:03:16.374 INFO:teuthology.provision.fog.smithi013:Deploy complete! 2020-11-16T20:03:16.375 DEBUG:teuthology.parallel:result is None 2020-11-16T20:04:01.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:04:01.401 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-16T20:04:01.402 INFO:teuthology.orchestra.run.smithi092:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-16T20:04:01.803 INFO:teuthology.provision.fog.smithi092:Node is ready 2020-11-16T20:04:01.803 INFO:teuthology.orchestra.run.smithi092:> hostname 2020-11-16T20:04:01.849 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com 2020-11-16T20:04:01.850 INFO:teuthology.orchestra.run.smithi092:> grep smithi092.front.sepia.ceph.com /etc/hosts 2020-11-16T20:04:01.898 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092 2020-11-16T20:04:01.899 INFO:teuthology.orchestra.run.smithi092:> sudo hostname smithi092 2020-11-16T20:04:02.004 INFO:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/smithi092.front.sepia.ceph.com/smithi092/g' /etc/hosts 2020-11-16T20:04:02.020 INFO:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/172.21.15.92/172.21.15.92/g' /etc/hosts 2020-11-16T20:04:02.073 INFO:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/smithi092.front.sepia.ceph.com/smithi092/g' /etc/hostname 2020-11-16T20:04:02.130 INFO:teuthology.orchestra.run.smithi092:> sudo hostname smithi092 2020-11-16T20:04:02.185 INFO:teuthology.provision.fog.smithi092:Deploy complete! 2020-11-16T20:04:02.186 DEBUG:teuthology.parallel:result is None 2020-11-16T20:04:06.462 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com 2020-11-16T20:04:06.463 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T20:04:06.484 INFO:teuthology.lock.ops:Checking smithi013.front.sepia.ceph.com 2020-11-16T20:04:06.484 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-16T20:04:06.500 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server 2020-11-16T20:04:06.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T20:04:06.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:04:06.575 INFO:teuthology.orchestra.run.smithi092:> true 2020-11-16T20:04:06.799 INFO:teuthology.orchestra.run.smithi092:> uname -m 2020-11-16T20:04:06.845 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2020-11-16T20:04:06.846 INFO:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2020-11-16T20:04:06.894 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Ubuntu" 2020-11-16T20:04:06.894 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T20:04:06.894 INFO:teuthology.orchestra.run.smithi092.stdout:ID=ubuntu 2020-11-16T20:04:06.894 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE=debian 2020-11-16T20:04:06.894 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T20:04:06.895 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="18.04" 2020-11-16T20:04:06.895 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T20:04:06.895 INFO:teuthology.orchestra.run.smithi092.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T20:04:06.895 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T20:04:06.895 INFO:teuthology.orchestra.run.smithi092.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T20:04:06.895 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_CODENAME=bionic 2020-11-16T20:04:06.895 INFO:teuthology.orchestra.run.smithi092.stdout:UBUNTU_CODENAME=bionic 2020-11-16T20:04:06.896 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2020-11-16T20:04:06.913 INFO:teuthology.lock.ops:Updating [smithi013.front.sepia.ceph.com]: set os type and version on server 2020-11-16T20:04:06.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T20:04:06.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:04:06.983 INFO:teuthology.orchestra.run.smithi013:> true 2020-11-16T20:04:07.214 INFO:teuthology.orchestra.run.smithi013:> uname -m 2020-11-16T20:04:07.263 INFO:teuthology.orchestra.run.smithi013.stdout:x86_64 2020-11-16T20:04:07.264 INFO:teuthology.orchestra.run.smithi013:> cat /etc/os-release 2020-11-16T20:04:07.312 INFO:teuthology.orchestra.run.smithi013.stdout:NAME="Ubuntu" 2020-11-16T20:04:07.312 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T20:04:07.312 INFO:teuthology.orchestra.run.smithi013.stdout:ID=ubuntu 2020-11-16T20:04:07.312 INFO:teuthology.orchestra.run.smithi013.stdout:ID_LIKE=debian 2020-11-16T20:04:07.313 INFO:teuthology.orchestra.run.smithi013.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T20:04:07.313 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_ID="18.04" 2020-11-16T20:04:07.313 INFO:teuthology.orchestra.run.smithi013.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T20:04:07.313 INFO:teuthology.orchestra.run.smithi013.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T20:04:07.313 INFO:teuthology.orchestra.run.smithi013.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T20:04:07.313 INFO:teuthology.orchestra.run.smithi013.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T20:04:07.313 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_CODENAME=bionic 2020-11-16T20:04:07.314 INFO:teuthology.orchestra.run.smithi013.stdout:UBUNTU_CODENAME=bionic 2020-11-16T20:04:07.314 INFO:teuthology.lock.ops:Updating smithi013.front.sepia.ceph.com on lock server 2020-11-16T20:04:07.331 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-11-16T20:04:07.365 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi013.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCv1BfmhdKqKp0UJPrCQbmfkTwciMfmtzxUKy2fa+G04S2O/6QQAgY5Du51Q5EnMe1FWmE+2lwJiTCVIt8tVaFCjiMcHDOSWap5N6BtHMR2ylsQrKVy+8esDehTiiT9BKHRY+5rk/wadBy+NhTVFgUQKZQZOt/iHQhNtr/89CTDBttDO3VOpIQOYqhOYBdPM7FO/FQ2VxhX6TT+sO6dJZ9VdVIpSsXHyzVK7wvRgzJ7JuOIy68wM8GBu9PLMZ9+GgvImX6SAw0PPT2fACDBymSBIEGogJXqvdsU8sRbhT2OQJQOLaT7doWCjMMIBmegEDraQ8VrRxJsjh0wrIn59823 smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKWHue7qo4kG7vXGAjFr6xMwF4Q1TyjEPt99U/vtRzjOsmbiEb4jSr7bxB1LcmqKR3oXnLUN69JwsszKPNfgI8BXSofGk3OrQisrBcaXFbeuzulKMSGGfFQYrNFiacFETOkbL6D9r9qhyNQZegacGZPCy67Wb3u82T8IqpiWcN+v3UkOB+t0SOGkdAMs1rUxmZmuGc+bFSS9c1TRyRrohoWfT9xi9VrUN2u7CJ7h9M+yDxIHhMxM9zxgo3vct6qY/htTNo8PQ3CG/esEykxy0lS8Sud2+we2fXph/weWmJhIw3VrWZrxBrwbdxOk8owIf657pE5q5J4xpztJLg9Bax 2020-11-16T20:04:07.365 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-16T20:04:07.417 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-16T20:04:07.435 INFO:teuthology.task.internal:Saving configuration 2020-11-16T20:04:07.451 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-16T20:04:07.459 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-16T20:04:07.474 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 19:58:25.896618', 'description': '/home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631141', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC6sfks32p5A1YhYoNtsckK06plx5kGVU/YGhGnVPAWG', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi092.front.sepia.ceph.com'} 2020-11-16T20:04:07.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-16 19:58:25.896648', 'description': '/home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631141', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJMGKSOT2SPfylLL5pBh2+WCpZbaWdCJcDMwn4l5Pfdr', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi013.front.sepia.ceph.com'} 2020-11-16T20:04:07.490 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-16T20:04:07.503 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2020-11-16T20:04:07.503 INFO:teuthology.task.internal:roles: ubuntu@smithi013.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2020-11-16T20:04:07.503 INFO:teuthology.run_tasks:Running task console_log... 2020-11-16T20:04:11.720 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5a70e16c80>, signals=[15]) 2020-11-16T20:04:11.721 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-16T20:04:11.730 INFO:teuthology.task.internal:Opening connections... 2020-11-16T20:04:11.730 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2020-11-16T20:04:11.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:04:11.808 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T20:04:11.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:04:11.886 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-16T20:04:11.897 INFO:teuthology.orchestra.run.smithi092:> uname -m 2020-11-16T20:04:12.009 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2020-11-16T20:04:12.009 INFO:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2020-11-16T20:04:12.057 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Ubuntu" 2020-11-16T20:04:12.058 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T20:04:12.058 INFO:teuthology.orchestra.run.smithi092.stdout:ID=ubuntu 2020-11-16T20:04:12.058 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE=debian 2020-11-16T20:04:12.058 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T20:04:12.058 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="18.04" 2020-11-16T20:04:12.059 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T20:04:12.059 INFO:teuthology.orchestra.run.smithi092.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T20:04:12.059 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T20:04:12.059 INFO:teuthology.orchestra.run.smithi092.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T20:04:12.059 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_CODENAME=bionic 2020-11-16T20:04:12.059 INFO:teuthology.orchestra.run.smithi092.stdout:UBUNTU_CODENAME=bionic 2020-11-16T20:04:12.060 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2020-11-16T20:04:12.075 INFO:teuthology.orchestra.run.smithi013:> uname -m 2020-11-16T20:04:12.082 INFO:teuthology.orchestra.run.smithi013.stdout:x86_64 2020-11-16T20:04:12.084 INFO:teuthology.orchestra.run.smithi013:> cat /etc/os-release 2020-11-16T20:04:12.130 INFO:teuthology.orchestra.run.smithi013.stdout:NAME="Ubuntu" 2020-11-16T20:04:12.130 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-16T20:04:12.130 INFO:teuthology.orchestra.run.smithi013.stdout:ID=ubuntu 2020-11-16T20:04:12.131 INFO:teuthology.orchestra.run.smithi013.stdout:ID_LIKE=debian 2020-11-16T20:04:12.131 INFO:teuthology.orchestra.run.smithi013.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-16T20:04:12.132 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_ID="18.04" 2020-11-16T20:04:12.132 INFO:teuthology.orchestra.run.smithi013.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-16T20:04:12.132 INFO:teuthology.orchestra.run.smithi013.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-16T20:04:12.132 INFO:teuthology.orchestra.run.smithi013.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-16T20:04:12.132 INFO:teuthology.orchestra.run.smithi013.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-16T20:04:12.133 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_CODENAME=bionic 2020-11-16T20:04:12.133 INFO:teuthology.orchestra.run.smithi013.stdout:UBUNTU_CODENAME=bionic 2020-11-16T20:04:12.133 INFO:teuthology.lock.ops:Updating smithi013.front.sepia.ceph.com on lock server 2020-11-16T20:04:12.146 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-16T20:04:12.179 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-16T20:04:12.188 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-16T20:04:12.189 INFO:teuthology.orchestra.run.smithi013:> test '!' -e /home/ubuntu/cephtest 2020-11-16T20:04:12.191 INFO:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2020-11-16T20:04:12.196 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-16T20:04:12.207 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-16T20:04:12.207 INFO:teuthology.orchestra.run.smithi013:> test -z $(ls -A /var/lib/ceph) 2020-11-16T20:04:12.238 INFO:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2020-11-16T20:04:12.248 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-16T20:04:12.290 INFO:teuthology.run_tasks:Running task kernel... 2020-11-16T20:04:12.313 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'prometheus.a': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.b': {'kdb': True, 'sha1': 'distro'}} 2020-11-16T20:04:12.314 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-11-16T20:04:12.315 INFO:teuthology.orchestra.run.smithi092:> uname -r 2020-11-16T20:04:12.321 INFO:teuthology.orchestra.run.smithi092.stdout:4.15.0-123-generic 2020-11-16T20:04:12.322 INFO:teuthology.task.kernel:Running kernel on smithi092: 4.15.0-123-generic 2020-11-16T20:04:12.322 INFO:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2020-11-16T20:04:12.447 INFO:teuthology.orchestra.run.smithi092:> sudo apt-get update 2020-11-16T20:04:12.576 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-11-16T20:04:12.718 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T20:04:12.798 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-16T20:04:13.061 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-16T20:04:13.204 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,761 kB] 2020-11-16T20:04:13.618 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,162 kB] 2020-11-16T20:04:13.653 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [373 kB] 2020-11-16T20:04:13.665 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [17.4 kB] 2020-11-16T20:04:13.665 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [196 kB] 2020-11-16T20:04:13.671 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [26.3 kB] 2020-11-16T20:04:13.672 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,547 kB] 2020-11-16T20:04:13.770 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,692 kB] 2020-11-16T20:04:13.820 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [355 kB] 2020-11-16T20:04:13.827 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.4 kB] 2020-11-16T20:04:13.827 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [36.2 kB] 2020-11-16T20:04:13.828 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [7,308 B] 2020-11-16T20:04:14.597 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 7,439 kB in 2s (3,859 kB/s) 2020-11-16T20:04:16.117 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2020-11-16T20:04:16.164 INFO:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-11-16T20:04:16.243 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2020-11-16T20:04:16.476 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2020-11-16T20:04:16.477 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2020-11-16T20:04:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:E: Unable to locate package linux-image-current-generic 2020-11-16T20:04:16.556 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-11-16T20:04:16.556 INFO:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-11-16T20:04:16.629 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2020-11-16T20:04:16.855 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2020-11-16T20:04:16.856 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2020-11-16T20:04:17.026 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2020-11-16T20:04:17.026 INFO:teuthology.orchestra.run.smithi092.stdout: linux-generic linux-headers-4.15.0-124 linux-headers-4.15.0-124-generic 2020-11-16T20:04:17.027 INFO:teuthology.orchestra.run.smithi092.stdout: linux-headers-generic linux-image-4.15.0-124-generic 2020-11-16T20:04:17.027 INFO:teuthology.orchestra.run.smithi092.stdout: linux-modules-4.15.0-124-generic linux-modules-extra-4.15.0-124-generic 2020-11-16T20:04:17.028 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2020-11-16T20:04:17.028 INFO:teuthology.orchestra.run.smithi092.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-11-16T20:04:17.028 INFO:teuthology.orchestra.run.smithi092.stdout:Recommended packages: 2020-11-16T20:04:17.029 INFO:teuthology.orchestra.run.smithi092.stdout: thermald 2020-11-16T20:04:17.052 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2020-11-16T20:04:17.052 INFO:teuthology.orchestra.run.smithi092.stdout: linux-headers-4.15.0-124 linux-headers-4.15.0-124-generic 2020-11-16T20:04:17.052 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-4.15.0-124-generic linux-modules-4.15.0-124-generic 2020-11-16T20:04:17.053 INFO:teuthology.orchestra.run.smithi092.stdout: linux-modules-extra-4.15.0-124-generic 2020-11-16T20:04:17.054 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages will be upgraded: 2020-11-16T20:04:17.055 INFO:teuthology.orchestra.run.smithi092.stdout: linux-generic linux-headers-generic linux-image-generic 2020-11-16T20:04:17.232 INFO:teuthology.orchestra.run.smithi092.stdout:3 upgraded, 5 newly installed, 0 to remove and 1 not upgraded. 2020-11-16T20:04:17.232 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 65.9 MB of archives. 2020-11-16T20:04:17.232 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 333 MB of additional disk space will be used. 2020-11-16T20:04:17.232 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-124-generic amd64 4.15.0-124.127 [13.1 MB] 2020-11-16T20:04:18.226 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-124-generic amd64 4.15.0-124.127 [8,012 kB] 2020-11-16T20:04:18.473 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-124-generic amd64 4.15.0-124.127 [32.8 MB] 2020-11-16T20:04:19.446 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.124.111 [1,868 B] 2020-11-16T20:04:19.447 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.124.111 [2,624 B] 2020-11-16T20:04:19.447 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-124 all 4.15.0-124.127 [10.9 MB] 2020-11-16T20:04:19.752 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-124-generic amd64 4.15.0-124.127 [1,084 kB] 2020-11-16T20:04:19.781 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.124.111 [2,520 B] 2020-11-16T20:04:20.274 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 65.9 MB in 3s (24.3 MB/s) 2020-11-16T20:04:20.438 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-modules-4.15.0-124-generic. 2020-11-16T20:04:20.476 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-16T20:04:20.479 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../0-linux-modules-4.15.0-124-generic_4.15.0-124.127_amd64.deb ... 2020-11-16T20:04:20.523 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-modules-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:22.140 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-image-4.15.0-124-generic. 2020-11-16T20:04:22.152 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../1-linux-image-4.15.0-124-generic_4.15.0-124.127_amd64.deb ... 2020-11-16T20:04:22.247 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-image-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:22.735 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-124-generic. 2020-11-16T20:04:22.747 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-124-generic_4.15.0-124.127_amd64.deb ... 2020-11-16T20:04:22.777 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-modules-extra-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:26.995 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../3-linux-generic_4.15.0.124.111_amd64.deb ... 2020-11-16T20:04:27.139 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-generic (4.15.0.124.111) over (4.15.0.123.110) ... 2020-11-16T20:04:27.454 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.124.111_amd64.deb ... 2020-11-16T20:04:27.559 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-image-generic (4.15.0.124.111) over (4.15.0.123.110) ... 2020-11-16T20:04:27.896 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-headers-4.15.0-124. 2020-11-16T20:04:27.907 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../5-linux-headers-4.15.0-124_4.15.0-124.127_all.deb ... 2020-11-16T20:04:27.945 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-4.15.0-124 (4.15.0-124.127) ... 2020-11-16T20:04:33.014 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-headers-4.15.0-124-generic. 2020-11-16T20:04:33.036 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../6-linux-headers-4.15.0-124-generic_4.15.0-124.127_amd64.deb ... 2020-11-16T20:04:33.064 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:35.398 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.124.111_amd64.deb ... 2020-11-16T20:04:35.544 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-generic (4.15.0.124.111) over (4.15.0.123.110) ... 2020-11-16T20:04:35.872 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-modules-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:36.423 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-image-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:36.865 INFO:teuthology.orchestra.run.smithi092.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-124-generic 2020-11-16T20:04:36.865 INFO:teuthology.orchestra.run.smithi092.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-124-generic 2020-11-16T20:04:36.992 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-modules-extra-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:37.468 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-4.15.0-124 (4.15.0-124.127) ... 2020-11-16T20:04:37.594 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-image-generic (4.15.0.124.111) ... 2020-11-16T20:04:37.745 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:37.871 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-generic (4.15.0.124.111) ... 2020-11-16T20:04:37.997 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-generic (4.15.0.124.111) ... 2020-11-16T20:04:38.123 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for linux-image-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:04:38.335 INFO:teuthology.orchestra.run.smithi092.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-11-16T20:04:38.335 INFO:teuthology.orchestra.run.smithi092.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-124-generic 2020-11-16T20:04:53.693 INFO:teuthology.orchestra.run.smithi092.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-11-16T20:04:53.693 INFO:teuthology.orchestra.run.smithi092.stdout:Sourcing file `/etc/default/grub' 2020-11-16T20:04:53.727 INFO:teuthology.orchestra.run.smithi092.stdout:Generating grub configuration file ... 2020-11-16T20:04:54.059 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-11-16T20:04:54.093 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-11-16T20:04:54.434 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-16T20:04:54.467 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-16T20:04:55.867 INFO:teuthology.orchestra.run.smithi092.stdout:done 2020-11-16T20:04:56.274 INFO:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2020-11-16T20:04:56.296 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2020-11-16T20:04:56.296 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2020-11-16T20:04:56.297 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2020-11-16T20:04:56.297 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2020-11-16T20:04:56.297 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 16 2020-11-16T20:04:56.297 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2020-11-16T20:04:56.297 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2020-11-16T20:04:56.297 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2020-11-16T20:04:56.297 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 4.15.0.124.111 2020-11-16T20:04:56.298 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-11-16T20:04:56.298 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-4.15.0-124-generic, linux-modules-extra-4.15.0-124-generic, linux-firmware, intel-microcode, amd64-microcode 2020-11-16T20:04:56.298 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2020-11-16T20:04:56.298 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2020-11-16T20:04:56.298 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2020-11-16T20:04:56.298 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2020-11-16T20:04:56.299 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-123-generic Expected: 4.15.0-124-generic 2020-11-16T20:04:56.299 INFO:teuthology.orchestra.run.smithi013:> uname -r 2020-11-16T20:04:56.305 INFO:teuthology.orchestra.run.smithi013.stdout:4.15.0-123-generic 2020-11-16T20:04:56.305 INFO:teuthology.task.kernel:Running kernel on smithi013: 4.15.0-123-generic 2020-11-16T20:04:56.306 INFO:teuthology.orchestra.run.smithi013:> sudo apt-get clean 2020-11-16T20:04:56.426 INFO:teuthology.orchestra.run.smithi013:> sudo apt-get update 2020-11-16T20:04:56.554 INFO:teuthology.orchestra.run.smithi013.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-11-16T20:04:56.675 INFO:teuthology.orchestra.run.smithi013.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T20:04:56.754 INFO:teuthology.orchestra.run.smithi013.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-16T20:04:56.992 INFO:teuthology.orchestra.run.smithi013.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-16T20:04:57.107 INFO:teuthology.orchestra.run.smithi013.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,761 kB] 2020-11-16T20:04:57.428 INFO:teuthology.orchestra.run.smithi013.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,162 kB] 2020-11-16T20:04:57.522 INFO:teuthology.orchestra.run.smithi013.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [373 kB] 2020-11-16T20:04:57.526 INFO:teuthology.orchestra.run.smithi013.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [17.4 kB] 2020-11-16T20:04:57.526 INFO:teuthology.orchestra.run.smithi013.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [196 kB] 2020-11-16T20:04:57.529 INFO:teuthology.orchestra.run.smithi013.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [26.3 kB] 2020-11-16T20:04:57.529 INFO:teuthology.orchestra.run.smithi013.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,692 kB] 2020-11-16T20:04:57.648 INFO:teuthology.orchestra.run.smithi013.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,547 kB] 2020-11-16T20:04:57.720 INFO:teuthology.orchestra.run.smithi013.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [355 kB] 2020-11-16T20:04:57.724 INFO:teuthology.orchestra.run.smithi013.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [14.4 kB] 2020-11-16T20:04:57.724 INFO:teuthology.orchestra.run.smithi013.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [36.2 kB] 2020-11-16T20:04:57.724 INFO:teuthology.orchestra.run.smithi013.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [7,308 B] 2020-11-16T20:04:58.530 INFO:teuthology.orchestra.run.smithi013.stdout:Fetched 7,439 kB in 2s (3,963 kB/s) 2020-11-16T20:05:00.105 INFO:teuthology.orchestra.run.smithi013.stdout:Reading package lists... 2020-11-16T20:05:00.153 INFO:teuthology.orchestra.run.smithi013:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-11-16T20:05:00.233 INFO:teuthology.orchestra.run.smithi013.stdout:Reading package lists... 2020-11-16T20:05:00.481 INFO:teuthology.orchestra.run.smithi013.stdout:Building dependency tree... 2020-11-16T20:05:00.481 INFO:teuthology.orchestra.run.smithi013.stdout:Reading state information... 2020-11-16T20:05:00.566 INFO:teuthology.orchestra.run.smithi013.stderr:E: Unable to locate package linux-image-current-generic 2020-11-16T20:05:00.568 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-11-16T20:05:00.568 INFO:teuthology.orchestra.run.smithi013:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-11-16T20:05:00.663 INFO:teuthology.orchestra.run.smithi013.stdout:Reading package lists... 2020-11-16T20:05:00.972 INFO:teuthology.orchestra.run.smithi013.stdout:Building dependency tree... 2020-11-16T20:05:00.973 INFO:teuthology.orchestra.run.smithi013.stdout:Reading state information... 2020-11-16T20:05:01.186 INFO:teuthology.orchestra.run.smithi013.stdout:The following additional packages will be installed: 2020-11-16T20:05:01.186 INFO:teuthology.orchestra.run.smithi013.stdout: linux-generic linux-headers-4.15.0-124 linux-headers-4.15.0-124-generic 2020-11-16T20:05:01.187 INFO:teuthology.orchestra.run.smithi013.stdout: linux-headers-generic linux-image-4.15.0-124-generic 2020-11-16T20:05:01.187 INFO:teuthology.orchestra.run.smithi013.stdout: linux-modules-4.15.0-124-generic linux-modules-extra-4.15.0-124-generic 2020-11-16T20:05:01.188 INFO:teuthology.orchestra.run.smithi013.stdout:Suggested packages: 2020-11-16T20:05:01.188 INFO:teuthology.orchestra.run.smithi013.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2020-11-16T20:05:01.189 INFO:teuthology.orchestra.run.smithi013.stdout:Recommended packages: 2020-11-16T20:05:01.189 INFO:teuthology.orchestra.run.smithi013.stdout: thermald 2020-11-16T20:05:01.216 INFO:teuthology.orchestra.run.smithi013.stdout:The following NEW packages will be installed: 2020-11-16T20:05:01.217 INFO:teuthology.orchestra.run.smithi013.stdout: linux-headers-4.15.0-124 linux-headers-4.15.0-124-generic 2020-11-16T20:05:01.217 INFO:teuthology.orchestra.run.smithi013.stdout: linux-image-4.15.0-124-generic linux-modules-4.15.0-124-generic 2020-11-16T20:05:01.217 INFO:teuthology.orchestra.run.smithi013.stdout: linux-modules-extra-4.15.0-124-generic 2020-11-16T20:05:01.219 INFO:teuthology.orchestra.run.smithi013.stdout:The following packages will be upgraded: 2020-11-16T20:05:01.219 INFO:teuthology.orchestra.run.smithi013.stdout: linux-generic linux-headers-generic linux-image-generic 2020-11-16T20:05:01.396 INFO:teuthology.orchestra.run.smithi013.stdout:3 upgraded, 5 newly installed, 0 to remove and 1 not upgraded. 2020-11-16T20:05:01.396 INFO:teuthology.orchestra.run.smithi013.stdout:Need to get 65.9 MB of archives. 2020-11-16T20:05:01.396 INFO:teuthology.orchestra.run.smithi013.stdout:After this operation, 333 MB of additional disk space will be used. 2020-11-16T20:05:01.396 INFO:teuthology.orchestra.run.smithi013.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-124-generic amd64 4.15.0-124.127 [13.1 MB] 2020-11-16T20:05:02.465 INFO:teuthology.orchestra.run.smithi013.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-124-generic amd64 4.15.0-124.127 [8,012 kB] 2020-11-16T20:05:02.701 INFO:teuthology.orchestra.run.smithi013.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-124-generic amd64 4.15.0-124.127 [32.8 MB] 2020-11-16T20:05:03.629 INFO:teuthology.orchestra.run.smithi013.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.124.111 [1,868 B] 2020-11-16T20:05:03.629 INFO:teuthology.orchestra.run.smithi013.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.124.111 [2,624 B] 2020-11-16T20:05:03.630 INFO:teuthology.orchestra.run.smithi013.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-124 all 4.15.0-124.127 [10.9 MB] 2020-11-16T20:05:03.952 INFO:teuthology.orchestra.run.smithi013.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-124-generic amd64 4.15.0-124.127 [1,084 kB] 2020-11-16T20:05:03.993 INFO:teuthology.orchestra.run.smithi013.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.124.111 [2,520 B] 2020-11-16T20:05:04.502 INFO:teuthology.orchestra.run.smithi013.stdout:Fetched 65.9 MB in 3s (23.9 MB/s) 2020-11-16T20:05:04.646 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package linux-modules-4.15.0-124-generic. 2020-11-16T20:05:04.680 INFO:teuthology.orchestra.run.smithi013.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102077 files and directories currently installed.) 2020-11-16T20:05:04.683 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../0-linux-modules-4.15.0-124-generic_4.15.0-124.127_amd64.deb ... 2020-11-16T20:05:04.731 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking linux-modules-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:06.406 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package linux-image-4.15.0-124-generic. 2020-11-16T20:05:06.418 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../1-linux-image-4.15.0-124-generic_4.15.0-124.127_amd64.deb ... 2020-11-16T20:05:06.509 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking linux-image-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:07.025 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-124-generic. 2020-11-16T20:05:07.037 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-124-generic_4.15.0-124.127_amd64.deb ... 2020-11-16T20:05:07.075 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking linux-modules-extra-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:10.776 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../3-linux-generic_4.15.0.124.111_amd64.deb ... 2020-11-16T20:05:10.877 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking linux-generic (4.15.0.124.111) over (4.15.0.123.110) ... 2020-11-16T20:05:11.175 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.124.111_amd64.deb ... 2020-11-16T20:05:11.298 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking linux-image-generic (4.15.0.124.111) over (4.15.0.123.110) ... 2020-11-16T20:05:11.633 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package linux-headers-4.15.0-124. 2020-11-16T20:05:11.646 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../5-linux-headers-4.15.0-124_4.15.0-124.127_all.deb ... 2020-11-16T20:05:11.683 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking linux-headers-4.15.0-124 (4.15.0-124.127) ... 2020-11-16T20:05:16.505 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package linux-headers-4.15.0-124-generic. 2020-11-16T20:05:16.527 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../6-linux-headers-4.15.0-124-generic_4.15.0-124.127_amd64.deb ... 2020-11-16T20:05:16.572 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking linux-headers-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:18.974 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.124.111_amd64.deb ... 2020-11-16T20:05:19.090 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking linux-headers-generic (4.15.0.124.111) over (4.15.0.123.110) ... 2020-11-16T20:05:19.426 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up linux-modules-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:19.995 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up linux-image-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:20.473 INFO:teuthology.orchestra.run.smithi013.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-124-generic 2020-11-16T20:05:20.473 INFO:teuthology.orchestra.run.smithi013.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-124-generic 2020-11-16T20:05:20.597 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up linux-modules-extra-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:21.115 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up linux-headers-4.15.0-124 (4.15.0-124.127) ... 2020-11-16T20:05:21.266 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up linux-image-generic (4.15.0.124.111) ... 2020-11-16T20:05:21.392 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up linux-headers-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:21.535 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up linux-headers-generic (4.15.0.124.111) ... 2020-11-16T20:05:21.661 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up linux-generic (4.15.0.124.111) ... 2020-11-16T20:05:21.787 INFO:teuthology.orchestra.run.smithi013.stdout:Processing triggers for linux-image-4.15.0-124-generic (4.15.0-124.127) ... 2020-11-16T20:05:21.997 INFO:teuthology.orchestra.run.smithi013.stdout:/etc/kernel/postinst.d/initramfs-tools: 2020-11-16T20:05:21.997 INFO:teuthology.orchestra.run.smithi013.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-124-generic 2020-11-16T20:05:36.894 INFO:teuthology.orchestra.run.smithi013.stdout:/etc/kernel/postinst.d/zz-update-grub: 2020-11-16T20:05:36.895 INFO:teuthology.orchestra.run.smithi013.stdout:Sourcing file `/etc/default/grub' 2020-11-16T20:05:36.922 INFO:teuthology.orchestra.run.smithi013.stdout:Generating grub configuration file ... 2020-11-16T20:05:37.259 INFO:teuthology.orchestra.run.smithi013.stdout:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-11-16T20:05:37.282 INFO:teuthology.orchestra.run.smithi013.stdout:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-11-16T20:05:37.607 INFO:teuthology.orchestra.run.smithi013.stdout:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-16T20:05:37.633 INFO:teuthology.orchestra.run.smithi013.stdout:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-16T20:05:38.603 INFO:teuthology.orchestra.run.smithi013.stdout:done 2020-11-16T20:05:38.983 INFO:teuthology.orchestra.run.smithi013:> dpkg -s linux-image-generic 2020-11-16T20:05:39.007 INFO:teuthology.orchestra.run.smithi013.stdout:Package: linux-image-generic 2020-11-16T20:05:39.007 INFO:teuthology.orchestra.run.smithi013.stdout:Status: install ok installed 2020-11-16T20:05:39.007 INFO:teuthology.orchestra.run.smithi013.stdout:Priority: optional 2020-11-16T20:05:39.007 INFO:teuthology.orchestra.run.smithi013.stdout:Section: kernel 2020-11-16T20:05:39.007 INFO:teuthology.orchestra.run.smithi013.stdout:Installed-Size: 16 2020-11-16T20:05:39.008 INFO:teuthology.orchestra.run.smithi013.stdout:Maintainer: Ubuntu Kernel Team 2020-11-16T20:05:39.008 INFO:teuthology.orchestra.run.smithi013.stdout:Architecture: amd64 2020-11-16T20:05:39.008 INFO:teuthology.orchestra.run.smithi013.stdout:Source: linux-meta 2020-11-16T20:05:39.008 INFO:teuthology.orchestra.run.smithi013.stdout:Version: 4.15.0.124.111 2020-11-16T20:05:39.008 INFO:teuthology.orchestra.run.smithi013.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-11-16T20:05:39.008 INFO:teuthology.orchestra.run.smithi013.stdout:Depends: linux-image-4.15.0-124-generic, linux-modules-extra-4.15.0-124-generic, linux-firmware, intel-microcode, amd64-microcode 2020-11-16T20:05:39.008 INFO:teuthology.orchestra.run.smithi013.stdout:Recommends: thermald 2020-11-16T20:05:39.009 INFO:teuthology.orchestra.run.smithi013.stdout:Description: Generic Linux kernel image 2020-11-16T20:05:39.009 INFO:teuthology.orchestra.run.smithi013.stdout: This package will always depend on the latest generic kernel image 2020-11-16T20:05:39.009 INFO:teuthology.orchestra.run.smithi013.stdout: available. 2020-11-16T20:05:39.009 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-123-generic Expected: 4.15.0-124-generic 2020-11-16T20:05:39.009 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2020-11-16T20:05:39.010 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-11-16T20:05:39.010 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-11-16T20:05:39.010 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2020-11-16T20:05:39.010 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2020-11-16T20:05:39.010 INFO:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2020-11-16T20:05:39.055 INFO:teuthology.orchestra.run.smithi092:> sudo apt-get update 2020-11-16T20:05:39.189 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T20:05:39.299 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T20:05:39.377 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-16T20:05:39.460 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-16T20:05:41.040 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2020-11-16T20:05:41.086 INFO:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-11-16T20:05:41.164 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2020-11-16T20:05:41.403 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2020-11-16T20:05:41.404 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2020-11-16T20:05:41.483 INFO:teuthology.orchestra.run.smithi092.stderr:E: Unable to locate package linux-image-current-generic 2020-11-16T20:05:41.484 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-11-16T20:05:41.484 INFO:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-11-16T20:05:41.560 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2020-11-16T20:05:41.786 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2020-11-16T20:05:41.787 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2020-11-16T20:05:41.979 INFO:teuthology.orchestra.run.smithi092.stdout:linux-image-generic is already the newest version (4.15.0.124.111). 2020-11-16T20:05:41.979 INFO:teuthology.orchestra.run.smithi092.stdout:0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded. 2020-11-16T20:05:41.981 INFO:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2020-11-16T20:05:42.002 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2020-11-16T20:05:42.002 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2020-11-16T20:05:42.002 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2020-11-16T20:05:42.003 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2020-11-16T20:05:42.003 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 16 2020-11-16T20:05:42.003 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2020-11-16T20:05:42.003 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2020-11-16T20:05:42.003 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2020-11-16T20:05:42.003 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 4.15.0.124.111 2020-11-16T20:05:42.003 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-11-16T20:05:42.004 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-4.15.0-124-generic, linux-modules-extra-4.15.0-124-generic, linux-firmware, intel-microcode, amd64-microcode 2020-11-16T20:05:42.004 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2020-11-16T20:05:42.004 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2020-11-16T20:05:42.004 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2020-11-16T20:05:42.004 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2020-11-16T20:05:42.005 INFO:teuthology.orchestra.run.smithi092:> mktemp 2020-11-16T20:05:42.050 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.ExC0zWMfLt 2020-11-16T20:05:42.051 INFO:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ExC0zWMfLt 2020-11-16T20:05:42.106 INFO:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.ExC0zWMfLt 2020-11-16T20:05:42.217 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.ExC0zWMfLt is 8KB 2020-11-16T20:05:42.226 INFO:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.ExC0zWMfLt 2020-11-16T20:05:42.232 INFO:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-11-16T20:05:42.287 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:05:42.288 INFO:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-11-16T20:05:42.288 INFO:teuthology.orchestra.run.smithi092:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-11-16T20:05:42.333 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-11-16T20:05:42.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:05:42.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:05:42.348 INFO:teuthology.orchestra.run.smithi092.stderr:93 bytes copied, 3.0948e-05 s, 3.0 MB/s 2020-11-16T20:05:42.349 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-11-16T20:05:42.358 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-124-generic 2020-11-16T20:05:42.358 INFO:teuthology.orchestra.run.smithi092:> sudo update-grub 2020-11-16T20:05:43.616 INFO:teuthology.orchestra.run.smithi092.stderr:Sourcing file `/etc/default/grub' 2020-11-16T20:05:43.638 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2020-11-16T20:05:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-11-16T20:05:43.881 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-11-16T20:05:44.185 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-16T20:05:44.198 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-16T20:05:45.266 INFO:teuthology.orchestra.run.smithi092.stderr:done 2020-11-16T20:05:45.267 INFO:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2020-11-16T20:05:45.269 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2020-11-16T20:05:45.270 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi013.front.sepia.ceph.com, path=None, version=distro) 2020-11-16T20:05:45.270 INFO:teuthology.orchestra.run.smithi013:> sudo apt-get clean 2020-11-16T20:05:45.311 INFO:teuthology.orchestra.run.smithi013:> sudo apt-get update 2020-11-16T20:05:45.500 INFO:teuthology.orchestra.run.smithi013.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T20:05:45.600 INFO:teuthology.orchestra.run.smithi013.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T20:05:45.686 INFO:teuthology.orchestra.run.smithi013.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2020-11-16T20:05:45.764 INFO:teuthology.orchestra.run.smithi013.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2020-11-16T20:05:47.493 INFO:teuthology.orchestra.run.smithi013.stdout:Reading package lists... 2020-11-16T20:05:47.543 INFO:teuthology.orchestra.run.smithi013:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2020-11-16T20:05:47.622 INFO:teuthology.orchestra.run.smithi013.stdout:Reading package lists... 2020-11-16T20:05:47.888 INFO:teuthology.orchestra.run.smithi013.stdout:Building dependency tree... 2020-11-16T20:05:47.888 INFO:teuthology.orchestra.run.smithi013.stdout:Reading state information... 2020-11-16T20:05:47.972 INFO:teuthology.orchestra.run.smithi013.stderr:E: Unable to locate package linux-image-current-generic 2020-11-16T20:05:47.974 DEBUG:teuthology.orchestra.run:got remote process result: 100 2020-11-16T20:05:47.974 INFO:teuthology.orchestra.run.smithi013:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2020-11-16T20:05:48.052 INFO:teuthology.orchestra.run.smithi013.stdout:Reading package lists... 2020-11-16T20:05:48.313 INFO:teuthology.orchestra.run.smithi013.stdout:Building dependency tree... 2020-11-16T20:05:48.314 INFO:teuthology.orchestra.run.smithi013.stdout:Reading state information... 2020-11-16T20:05:48.521 INFO:teuthology.orchestra.run.smithi013.stdout:linux-image-generic is already the newest version (4.15.0.124.111). 2020-11-16T20:05:48.522 INFO:teuthology.orchestra.run.smithi013.stdout:0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded. 2020-11-16T20:05:48.523 INFO:teuthology.orchestra.run.smithi013:> dpkg -s linux-image-generic 2020-11-16T20:05:48.546 INFO:teuthology.orchestra.run.smithi013.stdout:Package: linux-image-generic 2020-11-16T20:05:48.547 INFO:teuthology.orchestra.run.smithi013.stdout:Status: install ok installed 2020-11-16T20:05:48.547 INFO:teuthology.orchestra.run.smithi013.stdout:Priority: optional 2020-11-16T20:05:48.547 INFO:teuthology.orchestra.run.smithi013.stdout:Section: kernel 2020-11-16T20:05:48.547 INFO:teuthology.orchestra.run.smithi013.stdout:Installed-Size: 16 2020-11-16T20:05:48.547 INFO:teuthology.orchestra.run.smithi013.stdout:Maintainer: Ubuntu Kernel Team 2020-11-16T20:05:48.548 INFO:teuthology.orchestra.run.smithi013.stdout:Architecture: amd64 2020-11-16T20:05:48.548 INFO:teuthology.orchestra.run.smithi013.stdout:Source: linux-meta 2020-11-16T20:05:48.548 INFO:teuthology.orchestra.run.smithi013.stdout:Version: 4.15.0.124.111 2020-11-16T20:05:48.548 INFO:teuthology.orchestra.run.smithi013.stdout:Provides: wireguard-modules (= 1.0.20200611-1ubuntu1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2020-11-16T20:05:48.548 INFO:teuthology.orchestra.run.smithi013.stdout:Depends: linux-image-4.15.0-124-generic, linux-modules-extra-4.15.0-124-generic, linux-firmware, intel-microcode, amd64-microcode 2020-11-16T20:05:48.548 INFO:teuthology.orchestra.run.smithi013.stdout:Recommends: thermald 2020-11-16T20:05:48.548 INFO:teuthology.orchestra.run.smithi013.stdout:Description: Generic Linux kernel image 2020-11-16T20:05:48.549 INFO:teuthology.orchestra.run.smithi013.stdout: This package will always depend on the latest generic kernel image 2020-11-16T20:05:48.549 INFO:teuthology.orchestra.run.smithi013.stdout: available. 2020-11-16T20:05:48.549 INFO:teuthology.orchestra.run.smithi013:> mktemp 2020-11-16T20:05:48.591 INFO:teuthology.orchestra.run.smithi013.stdout:/tmp/tmp.ir5UICubLV 2020-11-16T20:05:48.591 INFO:teuthology.orchestra.run.smithi013:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ir5UICubLV 2020-11-16T20:05:48.639 INFO:teuthology.orchestra.run.smithi013:> sudo chmod 0666 /tmp/tmp.ir5UICubLV 2020-11-16T20:05:48.752 DEBUG:teuthology.orchestra.remote:smithi013:/tmp/tmp.ir5UICubLV is 8KB 2020-11-16T20:05:48.761 INFO:teuthology.orchestra.run.smithi013:> rm -fr /tmp/tmp.ir5UICubLV 2020-11-16T20:05:48.766 INFO:teuthology.orchestra.run.smithi013:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2020-11-16T20:05:48.821 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:05:48.821 INFO:teuthology.orchestra.run.smithi013:> sudo dd of=/etc/grub.d/01_ceph_kernel 2020-11-16T20:05:48.822 INFO:teuthology.orchestra.run.smithi013:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-11-16T20:05:48.867 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd of=/etc/grub.d/01_ceph_kernel 2020-11-16T20:05:48.880 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:05:48.880 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:05:48.880 INFO:teuthology.orchestra.run.smithi013.stderr:93 bytes copied, 2.9649e-05 s, 3.1 MB/s 2020-11-16T20:05:48.881 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo chmod 755 /etc/grub.d/01_ceph_kernel 2020-11-16T20:05:48.890 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-124-generic 2020-11-16T20:05:48.891 INFO:teuthology.orchestra.run.smithi013:> sudo update-grub 2020-11-16T20:05:50.232 INFO:teuthology.orchestra.run.smithi013.stderr:Sourcing file `/etc/default/grub' 2020-11-16T20:05:50.246 INFO:teuthology.orchestra.run.smithi013.stderr:Generating grub configuration file ... 2020-11-16T20:05:50.457 INFO:teuthology.orchestra.run.smithi013.stderr:Found linux image: /boot/vmlinuz-4.15.0-124-generic 2020-11-16T20:05:50.471 INFO:teuthology.orchestra.run.smithi013.stderr:Found initrd image: /boot/initrd.img-4.15.0-124-generic 2020-11-16T20:05:50.811 INFO:teuthology.orchestra.run.smithi013.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-16T20:05:50.823 INFO:teuthology.orchestra.run.smithi013.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-16T20:05:51.527 INFO:teuthology.orchestra.run.smithi013.stderr:done 2020-11-16T20:05:51.529 INFO:teuthology.orchestra.run.smithi013:> sudo shutdown -r now 2020-11-16T20:06:21.558 INFO:teuthology.misc:Re-opening connections... 2020-11-16T20:06:21.558 INFO:teuthology.misc:trying to connect to ubuntu@smithi092.front.sepia.ceph.com 2020-11-16T20:06:21.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T20:06:21.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:06:39.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2020-11-16T20:06:39.935 INFO:teuthology.misc:trying to connect to ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T20:06:39.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T20:06:39.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:07:39.938 DEBUG:teuthology.orchestra.remote:timed out 2020-11-16T20:07:39.940 DEBUG:teuthology.misc:waited 78.38194513320923 2020-11-16T20:07:40.941 INFO:teuthology.misc:trying to connect to ubuntu@smithi092.front.sepia.ceph.com 2020-11-16T20:07:40.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T20:07:40.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:07:41.024 INFO:teuthology.orchestra.run.smithi092:> true 2020-11-16T20:07:41.395 DEBUG:teuthology.misc:waited 79.83702325820923 2020-11-16T20:07:42.397 INFO:teuthology.misc:trying to connect to ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T20:07:42.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T20:07:42.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:07:42.473 INFO:teuthology.orchestra.run.smithi013:> true 2020-11-16T20:07:42.832 DEBUG:teuthology.misc:waited 81.27307391166687 2020-11-16T20:07:43.832 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2020-11-16T20:07:43.833 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.15.0-124-generic"... 2020-11-16T20:07:43.833 INFO:teuthology.orchestra.run.smithi092:> uname -r 2020-11-16T20:07:43.839 INFO:teuthology.orchestra.run.smithi092.stdout:4.15.0-124-generic 2020-11-16T20:07:43.839 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-124-generic vs 4.15.0-124-generic 2020-11-16T20:07:43.840 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-11-16T20:07:43.840 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-11-16T20:07:43.840 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2020-11-16T20:07:43.840 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.15.0-124-generic"... 2020-11-16T20:07:43.841 INFO:teuthology.orchestra.run.smithi013:> uname -r 2020-11-16T20:07:43.846 INFO:teuthology.orchestra.run.smithi013.stdout:4.15.0-124-generic 2020-11-16T20:07:43.847 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-124-generic vs 4.15.0-124-generic 2020-11-16T20:07:43.847 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-11-16T20:07:43.847 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-11-16T20:07:44.848 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-11-16T20:07:44.849 INFO:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-16T20:07:44.885 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2020-11-16T20:07:44.905 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-11-16T20:07:44.905 INFO:teuthology.orchestra.run.smithi013:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-11-16T20:07:44.952 INFO:teuthology.orchestra.run.smithi013.stdout:ttyS1 2020-11-16T20:07:44.972 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-16T20:07:47.639 INFO:teuthology.task.internal:Creating test directory... 2020-11-16T20:07:47.640 INFO:teuthology.orchestra.run.smithi013:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T20:07:47.642 INFO:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-16T20:07:47.648 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-16T20:07:47.707 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-16T20:07:51.964 INFO:teuthology.task.internal:Creating archive directory... 2020-11-16T20:07:51.964 INFO:teuthology.orchestra.run.smithi013:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T20:07:51.969 INFO:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-16T20:07:51.997 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-16T20:07:52.078 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-16T20:07:52.079 INFO:teuthology.orchestra.run.smithi013:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T20:07:52.082 INFO:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-16T20:07:52.101 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T20:07:52.107 INFO:teuthology.orchestra.run.smithi013.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T20:07:52.111 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T20:07:52.116 INFO:teuthology.orchestra.run.smithi013.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-16T20:07:52.117 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-16T20:07:52.126 INFO:teuthology.task.internal:Configuring sudo... 2020-11-16T20:07:52.127 INFO:teuthology.orchestra.run.smithi013:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T20:07:52.160 INFO:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-16T20:07:52.176 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-16T20:07:52.187 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-16T20:07:52.187 INFO:teuthology.orchestra.run.smithi013:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T20:07:52.216 INFO:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-16T20:07:52.228 INFO:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T20:07:52.277 INFO:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T20:07:52.324 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:07:52.325 INFO:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T20:07:52.370 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T20:07:52.385 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:07:52.386 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:07:52.386 INFO:teuthology.orchestra.run.smithi092.stderr:152 bytes copied, 3.3471e-05 s, 4.5 MB/s 2020-11-16T20:07:52.387 INFO:teuthology.orchestra.run.smithi013:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-16T20:07:52.396 INFO:teuthology.orchestra.run.smithi013:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-16T20:07:52.444 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:07:52.444 INFO:teuthology.orchestra.run.smithi013:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T20:07:52.490 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-16T20:07:52.506 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:07:52.506 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:07:52.506 INFO:teuthology.orchestra.run.smithi013.stderr:152 bytes copied, 3.42e-05 s, 4.4 MB/s 2020-11-16T20:07:52.507 INFO:teuthology.orchestra.run.smithi013:> sudo service rsyslog restart 2020-11-16T20:07:52.553 INFO:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2020-11-16T20:07:52.604 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-16T20:07:52.612 INFO:teuthology.task.internal:Starting timer... 2020-11-16T20:07:52.612 INFO:teuthology.run_tasks:Running task pcp... 2020-11-16T20:07:52.623 INFO:teuthology.run_tasks:Running task selinux... 2020-11-16T20:07:52.633 DEBUG:teuthology.task.selinux:Excluding smithi092: OS 'ubuntu' does not support SELinux 2020-11-16T20:07:52.633 DEBUG:teuthology.task.selinux:Excluding smithi013: OS 'ubuntu' does not support SELinux 2020-11-16T20:07:52.634 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-16T20:07:52.634 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-11-16T20:07:52.634 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-16T20:07:52.634 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-16T20:07:52.645 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2020-11-16T20:07:52.652 INFO:teuthology.repo_utils:Fetching master from origin 2020-11-16T20:07:56.823 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-16T20:07:56.837 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-16T20:07:56.838 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi013.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-16T20:07:57.154 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-16T20:07:57.927 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-16T20:08:08.636 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-16T20:08:08.656 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:08 +0000 (0:00:00.070) 0:00:00.070 ******* 2020-11-16T20:08:08.670 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:08 +0000 (0:00:00.014) 0:00:00.084 ******* 2020-11-16T20:08:14.410 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi013.front.sepia.ceph.com] 2020-11-16T20:08:14.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:14 +0000 (0:00:05.813) 0:00:05.898 ******* 2020-11-16T20:08:18.695 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] 2020-11-16T20:08:18.765 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:18 +0000 (0:00:04.281) 0:00:10.179 ******* 2020-11-16T20:08:31.391 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-11-16T20:08:31.393 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T20:08:31.427 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:31 +0000 (0:00:12.661) 0:00:22.841 ******* 2020-11-16T20:08:39.821 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T20:08:39.855 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:39 +0000 (0:00:08.428) 0:00:31.270 ******* 2020-11-16T20:08:40.246 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-11-16T20:08:40.248 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-16T20:08:40.282 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:40 +0000 (0:00:00.426) 0:00:31.696 ******* 2020-11-16T20:08:40.289 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-16T20:08:40.323 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:40 +0000 (0:00:00.041) 0:00:31.738 ******* 2020-11-16T20:08:44.872 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-16T20:08:44.874 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.088507", "end": "2020-11-16 20:08:45.277328", "rc": 0, "start": "2020-11-16 20:08:45.188821", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-16T20:08:44.906 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:44 +0000 (0:00:04.583) 0:00:36.321 ******* 2020-11-16T20:08:45.160 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-16T20:08:45.161 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T20:08:45.196 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:45 +0000 (0:00:00.289) 0:00:36.611 ******* 2020-11-16T20:08:45.943 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-11-16T20:08:45.945 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605557325.2116168-181592628853047/source", "state": "file", "uid": 0} 2020-11-16T20:08:45.976 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:08:45 +0000 (0:00:00.780) 0:00:37.391 ******* 2020-11-16T20:08:46.282 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-16T20:08:46.283 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-16T20:08:46.285 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-16T20:09:05.892 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-16T20:09:05.894 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.085797", "end": "2020-11-16 20:09:05.874073", "rc": 0, "start": "2020-11-16 20:09:05.788276", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-16T20:09:05.924 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:09:05 +0000 (0:00:19.948) 0:00:57.339 ******* 2020-11-16T20:09:06.044 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-16T20:09:06.046 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T20:09:06.077 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:09:06 +0000 (0:00:00.152) 0:00:57.492 ******* 2020-11-16T20:09:06.463 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-11-16T20:09:06.465 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605557346.0925672-235475879330867/source", "state": "file", "uid": 0} 2020-11-16T20:09:06.499 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:09:06 +0000 (0:00:00.422) 0:00:57.914 ******* 2020-11-16T20:09:06.653 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-16T20:09:06.655 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-16T20:09:06.657 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-16T20:09:06.770 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-11-16T20:09:06.777 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-11-16T20:09:06.906 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:09:06 +0000 (0:00:00.406) 0:00:58.320 ******* 2020-11-16T20:09:19.703 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:09:19 +0000 (0:00:12.797) 0:01:11.118 ******* 2020-11-16T20:09:33.144 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi013.front.sepia.ceph.com] 2020-11-16T20:09:33.225 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-11-16T20:09:33.227 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:09:33 +0000 (0:00:13.522) 0:01:24.640 ******* 2020-11-16T20:09:33.362 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] 2020-11-16T20:09:33.443 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:09:33 +0000 (0:00:00.218) 0:01:24.858 ******* 2020-11-16T20:10:11.813 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-16T20:10:11.815 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:04.479590", "end": "2020-11-16 20:10:11.790594", "rc": 0, "start": "2020-11-16 20:10:07.311004", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T20:10:11.847 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:11 +0000 (0:00:38.404) 0:02:03.262 ******* 2020-11-16T20:10:11.988 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-16T20:10:11.990 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.005901", "end": "2020-11-16 20:10:12.354244", "failed_when_result": false, "rc": 0, "start": "2020-11-16 20:10:12.348343", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T20:10:12.022 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:12 +0000 (0:00:00.174) 0:02:03.436 ******* 2020-11-16T20:10:12.049 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-16T20:10:12.050 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-16T20:10:12.082 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:12 +0000 (0:00:00.059) 0:02:03.496 ******* 2020-11-16T20:10:12.108 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T20:10:12.109 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.143 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:12 +0000 (0:00:00.061) 0:02:03.558 ******* 2020-11-16T20:10:12.169 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-16T20:10:12.171 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.202 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:12 +0000 (0:00:00.058) 0:02:03.616 ******* 2020-11-16T20:10:12.228 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-16T20:10:12.229 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.261 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:12 +0000 (0:00:00.058) 0:02:03.675 ******* 2020-11-16T20:10:12.287 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-16T20:10:12.289 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.320 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:12 +0000 (0:00:00.059) 0:02:03.735 ******* 2020-11-16T20:10:12.347 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-16T20:10:12.349 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.380 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:12 +0000 (0:00:00.059) 0:02:03.795 ******* 2020-11-16T20:10:12.417 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T20:10:12.419 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.428 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.430 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.440 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.441 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.451 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.452 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.463 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.464 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.474 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.476 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.489 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.490 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.501 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.503 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.512 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.513 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.524 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.526 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.535 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.537 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.547 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.547 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.547 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.558 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.560 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.570 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.571 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.581 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.583 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.593 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.594 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.604 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.605 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.606 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.617 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.618 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.629 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.631 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.641 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.642 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.653 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.653 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.664 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.665 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.665 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.676 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.677 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.687 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.688 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.690 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.700 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.701 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.711 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.713 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.722 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.723 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.723 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.734 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.746 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.747 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.758 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.759 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.769 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.770 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.781 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.782 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.792 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.794 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.804 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.805 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.815 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.817 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.826 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.828 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.838 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.839 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.849 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.850 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.860 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.862 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.872 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.873 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.883 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.885 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.894 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.896 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.906 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.908 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.917 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.919 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.931 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.932 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.933 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.935 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.945 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.946 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.957 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.959 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.969 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.971 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.981 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.982 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.992 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:12.994 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.004 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.006 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.015 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.017 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.026 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.028 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.038 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.040 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.050 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.051 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.053 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.063 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.064 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.076 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.077 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.087 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.089 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.099 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.158 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:13 +0000 (0:00:00.777) 0:02:04.572 ******* 2020-11-16T20:10:13.185 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-16T20:10:13.217 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:13 +0000 (0:00:00.059) 0:02:04.632 ******* 2020-11-16T20:10:13.245 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.283 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:13 +0000 (0:00:00.065) 0:02:04.697 ******* 2020-11-16T20:10:13.310 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-11-16T20:10:13.312 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.346 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:13 +0000 (0:00:00.063) 0:02:04.760 ******* 2020-11-16T20:10:13.372 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-16T20:10:13.374 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.407 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:13 +0000 (0:00:00.060) 0:02:04.821 ******* 2020-11-16T20:10:13.434 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-16T20:10:13.435 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.467 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:13 +0000 (0:00:00.060) 0:02:04.882 ******* 2020-11-16T20:10:13.494 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-16T20:10:13.496 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.529 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:13 +0000 (0:00:00.061) 0:02:04.943 ******* 2020-11-16T20:10:13.556 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T20:10:13.557 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.567 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.569 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.579 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.580 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.581 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.591 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.593 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.603 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.604 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.615 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.616 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.626 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.627 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.639 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.650 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.651 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.661 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.663 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.672 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.674 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.684 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.685 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.695 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.697 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.707 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.708 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.718 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.720 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.730 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.732 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.741 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.743 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.753 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.754 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.754 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.765 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.765 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.776 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.778 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-16T20:10:13.814 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:13 +0000 (0:00:00.285) 0:02:05.229 ******* 2020-11-16T20:10:18.626 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-16T20:10:18.627 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-11-16T20:10:18.659 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:18 +0000 (0:00:04.844) 0:02:10.073 ******* 2020-11-16T20:10:18.686 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-16T20:10:18.687 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => { "msg": "Host smithi013.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-11-16T20:10:18.719 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:18 +0000 (0:00:00.060) 0:02:10.134 ******* 2020-11-16T20:10:18.756 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-16T20:10:18.758 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-11-16T20:10:18.789 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:18 +0000 (0:00:00.069) 0:02:10.204 ******* 2020-11-16T20:10:18.816 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-16T20:10:18.818 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:18.849 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:18 +0000 (0:00:00.059) 0:02:10.264 ******* 2020-11-16T20:10:18.876 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T20:10:18.878 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:18.909 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:18 +0000 (0:00:00.059) 0:02:10.323 ******* 2020-11-16T20:10:18.935 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-16T20:10:18.937 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:18.978 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:18 +0000 (0:00:00.069) 0:02:10.392 ******* 2020-11-16T20:10:19.007 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-16T20:10:19.008 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.042 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.064) 0:02:10.457 ******* 2020-11-16T20:10:19.070 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-16T20:10:19.071 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.103 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.060) 0:02:10.517 ******* 2020-11-16T20:10:19.131 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.163 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.060) 0:02:10.578 ******* 2020-11-16T20:10:19.191 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T20:10:19.193 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.224 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.060) 0:02:10.638 ******* 2020-11-16T20:10:19.251 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-16T20:10:19.252 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.283 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.059) 0:02:10.697 ******* 2020-11-16T20:10:19.310 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-16T20:10:19.312 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi013.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:10:19.343 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.059) 0:02:10.757 ******* 2020-11-16T20:10:19.370 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-11-16T20:10:19.372 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.403 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.060) 0:02:10.818 ******* 2020-11-16T20:10:19.430 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-16T20:10:19.432 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.463 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.059) 0:02:10.877 ******* 2020-11-16T20:10:19.490 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi013.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:10:19.522 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.059) 0:02:10.937 ******* 2020-11-16T20:10:19.549 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-16T20:10:19.551 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.582 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.059) 0:02:10.997 ******* 2020-11-16T20:10:19.609 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-11-16T20:10:19.611 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:10:19.645 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.062) 0:02:11.059 ******* 2020-11-16T20:10:19.672 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-16T20:10:19.673 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.705 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.059) 0:02:11.119 ******* 2020-11-16T20:10:19.732 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-16T20:10:19.734 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.768 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.062) 0:02:11.182 ******* 2020-11-16T20:10:19.794 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-16T20:10:19.796 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.828 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.059) 0:02:11.242 ******* 2020-11-16T20:10:19.854 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-16T20:10:19.856 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:19.890 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.061) 0:02:11.304 ******* 2020-11-16T20:10:19.917 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-16T20:10:19.919 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:10:19.952 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:19 +0000 (0:00:00.062) 0:02:11.366 ******* 2020-11-16T20:10:19.978 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T20:10:19.980 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:20.012 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:20 +0000 (0:00:00.059) 0:02:11.426 ******* 2020-11-16T20:10:20.039 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-16T20:10:20.041 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:20.073 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:20 +0000 (0:00:00.061) 0:02:11.488 ******* 2020-11-16T20:10:20.100 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-16T20:10:20.102 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:20.134 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:20 +0000 (0:00:00.060) 0:02:11.548 ******* 2020-11-16T20:10:20.161 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:10:20.196 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:10:20 +0000 (0:00:00.062) 0:02:11.611 ******* 2020-11-16T20:11:23.995 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:24.029 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:24 +0000 (0:01:03.832) 0:03:15.443 ******* 2020-11-16T20:11:36.788 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-16T20:11:36.790 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.163231", "end": "2020-11-16 20:10:20.287203", "rc": 0, "start": "2020-11-16 20:10:20.123972", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T20:11:36.823 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:36 +0000 (0:00:12.794) 0:03:28.238 ******* 2020-11-16T20:11:36.829 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-16T20:11:36.831 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:36.864 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:36 +0000 (0:00:00.041) 0:03:28.279 ******* 2020-11-16T20:11:41.164 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-16T20:11:41.166 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:11:41.197 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:41 +0000 (0:00:04.332) 0:03:32.611 ******* 2020-11-16T20:11:53.974 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-16T20:11:53.976 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:54.011 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:54 +0000 (0:00:12.814) 0:03:45.425 ******* 2020-11-16T20:11:54.040 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-16T20:11:54.042 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:54.076 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:54 +0000 (0:00:00.065) 0:03:45.490 ******* 2020-11-16T20:11:54.104 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:54.137 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:54 +0000 (0:00:00.060) 0:03:45.551 ******* 2020-11-16T20:11:54.175 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T20:11:54.177 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:54.212 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:54 +0000 (0:00:00.073) 0:03:45.624 ******* 2020-11-16T20:11:54.251 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-16T20:11:54.253 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-16T20:11:54.303 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:54 +0000 (0:00:00.093) 0:03:45.718 ******* 2020-11-16T20:11:58.549 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-16T20:11:58.551 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.051418", "end": "2020-11-16 20:11:54.250913", "failed_when_result": false, "rc": 0, "start": "2020-11-16 20:11:54.199495", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-16T20:11:58.585 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:58 +0000 (0:00:04.281) 0:03:49.999 ******* 2020-11-16T20:11:58.591 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T20:11:58.593 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:58.626 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:58 +0000 (0:00:00.041) 0:03:50.040 ******* 2020-11-16T20:11:58.633 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-16T20:11:58.634 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-16T20:11:58.671 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:58 +0000 (0:00:00.045) 0:03:50.085 ******* 2020-11-16T20:11:58.699 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-16T20:11:58.701 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:58.735 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:58 +0000 (0:00:00.063) 0:03:50.149 ******* 2020-11-16T20:11:58.764 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:11:58.802 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:11:58 +0000 (0:00:00.066) 0:03:50.216 ******* 2020-11-16T20:12:03.079 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T20:12:03.081 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.015355", "end": "2020-11-16 20:11:59.027372", "rc": 0, "start": "2020-11-16 20:11:59.012017", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T20:12:03.114 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:03 +0000 (0:00:04.312) 0:03:54.529 ******* 2020-11-16T20:12:03.121 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-11-16T20:12:03.123 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:03.159 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:03 +0000 (0:00:00.045) 0:03:54.574 ******* 2020-11-16T20:12:03.167 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T20:12:03.169 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:03.202 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:03 +0000 (0:00:00.042) 0:03:54.617 ******* 2020-11-16T20:12:03.213 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-11-16T20:12:03.214 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:03.248 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:03 +0000 (0:00:00.045) 0:03:54.662 ******* 2020-11-16T20:12:03.255 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T20:12:03.257 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:03.294 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:03 +0000 (0:00:00.045) 0:03:54.708 ******* 2020-11-16T20:12:03.301 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-16T20:12:03.303 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:03.336 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:03 +0000 (0:00:00.042) 0:03:54.751 ******* 2020-11-16T20:12:19.892 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T20:12:19.894 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:19.927 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:19 +0000 (0:00:16.590) 0:04:11.342 ******* 2020-11-16T20:12:19.935 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:19.936 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:19.947 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:19.949 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone] *********************************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:19.981 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:19 +0000 (0:00:00.054) 0:04:11.396 ******* 2020-11-16T20:12:19.988 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:19.990 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:19.991 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f30 2020-11-16T20:12:19.993 INFO:teuthology.task.ansible.out:09adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6I 2020-11-16T20:12:19.994 INFO:teuthology.task.ansible.out:nynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.003 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.004 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d3430 2020-11-16T20:12:20.006 INFO:teuthology.task.ansible.out:0b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.037 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:20 +0000 (0:00:00.055) 0:04:11.452 ******* 2020-11-16T20:12:20.045 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.046 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.047 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.048 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96 2020-11-16T20:12:20.049 INFO:teuthology.task.ansible.out:a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f2 2020-11-16T20:12:20.051 INFO:teuthology.task.ansible.out:7af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.060 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.062 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.094 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:20 +0000 (0:00:00.057) 0:04:11.509 ******* 2020-11-16T20:12:20.101 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-16T20:12:20.103 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJi 2020-11-16T20:12:20.105 INFO:teuthology.task.ansible.out:hFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.116 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.118 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} TASK [common : Refresh repos] ************************************************** 2020-11-16T20:12:20.119 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385af 2020-11-16T20:12:20.120 INFO:teuthology.task.ansible.out:f099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.152 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:20 +0000 (0:00:00.057) 0:04:11.566 ******* 2020-11-16T20:12:20.160 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.161 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.162 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.163 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.163 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.163 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.165 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.165 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.176 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.178 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.188 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.189 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.199 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.201 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.211 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.212 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.222 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.224 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.234 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.234 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.246 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.247 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.257 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.258 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.260 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.270 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.271 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.272 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.283 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.284 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.294 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.296 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.305 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.307 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.317 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.328 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.331 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.342 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.342 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.354 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.355 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.366 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.378 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.379 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.391 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.402 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.404 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.414 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.416 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.426 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.427 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.437 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.439 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.449 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.450 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.460 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.462 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.472 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.473 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.473 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.484 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.485 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.495 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.497 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.507 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.508 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.520 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.521 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} TASK [common : Set the hardware clock] ***************************************** changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.244201", "end": "2020-11-16 20:12:20.500411", "rc": 0, "start": "2020-11-16 20:12:20.256210", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:12:20.522 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.559 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:20 +0000 (0:00:00.406) 0:04:11.973 ******* 2020-11-16T20:12:20.566 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.567 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.568 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.568 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.570 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => 2020-11-16T20:12:20.571 INFO:teuthology.task.ansible.out:(item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.581 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.583 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Create all admin users with sudo access.] ************************ skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:20.616 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:20 +0000 (0:00:00.057) 0:04:12.030 ******* 2020-11-16T20:12:24.799 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:24.807 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:24.809 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dorinda', 'ovpn': 'dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"n 2020-11-16T20:12:24.811 INFO:teuthology.task.ansible.out:ame": "dorinda", "ovpn": "dorinda@ideapad /BLvrh5FMYG1lXzwak5Z+w d028380a40a95f162324dfbb024cfa98b5771a1b90741fbfc908cad93ef09a14"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:24.857 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:24 +0000 (0:00:04.241) 0:04:16.272 ******* 2020-11-16T20:12:24.885 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-16T20:12:24.917 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:24 +0000 (0:00:00.059) 0:04:16.332 ******* 2020-11-16T20:12:29.059 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-16T20:12:29.060 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.096 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:29 +0000 (0:00:04.179) 0:04:20.511 ******* 2020-11-16T20:12:29.103 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-16T20:12:29.105 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.054207", "end": "2020-11-16 20:12:29.063232", "rc": 0, "start": "2020-11-16 20:12:29.009025", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:12:29.141 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:29 +0000 (0:00:00.044) 0:04:20.555 ******* 2020-11-16T20:12:29.148 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-11-16T20:12:29.150 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.189 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:29 +0000 (0:00:00.047) 0:04:20.603 ******* 2020-11-16T20:12:29.216 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.250 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:29 +0000 (0:00:00.061) 0:04:20.664 ******* 2020-11-16T20:12:29.277 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-16T20:12:29.279 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.311 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:29 +0000 (0:00:00.061) 0:04:20.726 ******* 2020-11-16T20:12:29.339 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.372 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:29 +0000 (0:00:00.060) 0:04:20.787 ******* 2020-11-16T20:12:29.399 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-16T20:12:29.401 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.410 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.412 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.421 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.423 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.433 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.434 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.444 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.445 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.455 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.456 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.457 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.467 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.469 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.479 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.480 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.490 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.492 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.502 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.503 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.513 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.515 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.525 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.527 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.537 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.538 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.548 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.550 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.560 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.561 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.571 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.573 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.584 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.585 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.595 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.597 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.607 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.608 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-16T20:12:29.646 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:29 +0000 (0:00:00.273) 0:04:21.060 ******* 2020-11-16T20:12:29.953 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-16T20:12:29.955 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-11-16T20:12:29.989 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:29 +0000 (0:00:00.343) 0:04:21.404 ******* 2020-11-16T20:12:30.017 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-16T20:12:30.018 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => { "msg": "Host smithi092.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-11-16T20:12:30.050 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:30 +0000 (0:00:00.060) 0:04:21.465 ******* 2020-11-16T20:12:30.100 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-11-16T20:12:30.147 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:30 +0000 (0:00:00.096) 0:04:21.562 ******* 2020-11-16T20:12:34.315 INFO:teuthology.task.ansible.out: 2020-11-16T20:12:34.316 INFO:teuthology.task.ansible.out:TASK [common : Update apt cache] *********************************************** ok: [smithi013.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:12:34.351 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:34 +0000 (0:00:04.203) 0:04:25.765 ******* 2020-11-16T20:12:34.357 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-16T20:12:34.359 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:34.392 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:34 +0000 (0:00:00.041) 0:04:25.807 ******* 2020-11-16T20:12:34.419 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-16T20:12:34.421 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:34.452 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:34 +0000 (0:00:00.060) 0:04:25.867 ******* 2020-11-16T20:12:34.479 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-16T20:12:34.481 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:34.512 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:34 +0000 (0:00:00.059) 0:04:25.927 ******* 2020-11-16T20:12:34.538 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-16T20:12:34.540 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:34.571 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:34 +0000 (0:00:00.058) 0:04:25.986 ******* 2020-11-16T20:12:34.597 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-16T20:12:34.599 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:34.630 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:34 +0000 (0:00:00.059) 0:04:26.045 ******* 2020-11-16T20:12:34.657 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:34.690 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:34 +0000 (0:00:00.059) 0:04:26.105 ******* 2020-11-16T20:12:34.716 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-16T20:12:34.718 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:34.749 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:34 +0000 (0:00:00.059) 0:04:26.164 ******* 2020-11-16T20:12:38.909 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T20:12:38.911 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:12:38.944 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:38 +0000 (0:00:04.194) 0:04:30.359 ******* 2020-11-16T20:12:38.951 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-16T20:12:38.952 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:38.985 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:38 +0000 (0:00:00.040) 0:04:30.399 ******* 2020-11-16T20:12:38.991 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T20:12:38.993 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:12:39.025 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:39 +0000 (0:00:00.040) 0:04:30.440 ******* 2020-11-16T20:12:39.032 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-16T20:12:39.033 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:12:39.069 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:12:39 +0000 (0:00:00.042) 0:04:30.483 ******* 2020-11-16T20:15:20.636 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:20.669 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:20 +0000 (0:02:41.600) 0:07:12.083 ******* 2020-11-16T20:15:20.696 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:20.728 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:20 +0000 (0:00:00.059) 0:07:12.143 ******* 2020-11-16T20:15:20.756 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:15:20.788 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:20 +0000 (0:00:00.060) 0:07:12.203 ******* 2020-11-16T20:15:20.815 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-16T20:15:20.817 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:20.848 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:20 +0000 (0:00:00.059) 0:07:12.262 ******* 2020-11-16T20:15:20.875 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-11-16T20:15:20.877 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:15:20.908 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:20 +0000 (0:00:00.060) 0:07:12.323 ******* 2020-11-16T20:15:20.936 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-11-16T20:15:20.938 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:20.968 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:20 +0000 (0:00:00.060) 0:07:12.383 ******* 2020-11-16T20:15:29.303 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-16T20:15:29.305 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.337 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:08.368) 0:07:20.751 ******* 2020-11-16T20:15:29.364 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-16T20:15:29.366 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.397 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.060) 0:07:20.812 ******* 2020-11-16T20:15:29.425 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-16T20:15:29.426 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.457 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.060) 0:07:20.872 ******* 2020-11-16T20:15:29.485 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-16T20:15:29.487 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:15:29.517 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.060) 0:07:20.932 ******* 2020-11-16T20:15:29.545 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T20:15:29.547 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.577 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.059) 0:07:20.992 ******* 2020-11-16T20:15:29.605 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-16T20:15:29.606 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.637 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.060) 0:07:21.052 ******* 2020-11-16T20:15:29.668 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-11-16T20:15:29.670 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.700 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.062) 0:07:21.114 ******* 2020-11-16T20:15:29.726 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-16T20:15:29.728 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.758 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.058) 0:07:21.173 ******* 2020-11-16T20:15:29.784 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-16T20:15:29.786 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.816 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.058) 0:07:21.231 ******* 2020-11-16T20:15:29.843 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-16T20:15:29.845 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.875 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.059) 0:07:21.290 ******* 2020-11-16T20:15:29.902 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-16T20:15:29.904 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:15:29.934 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.058) 0:07:21.349 ******* 2020-11-16T20:15:29.961 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-16T20:15:29.962 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:29.993 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:29 +0000 (0:00:00.058) 0:07:21.407 ******* 2020-11-16T20:15:30.020 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-16T20:15:30.021 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:30.053 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:30 +0000 (0:00:00.060) 0:07:21.468 ******* 2020-11-16T20:15:30.080 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-16T20:15:30.082 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:30.113 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:30 +0000 (0:00:00.059) 0:07:21.527 ******* 2020-11-16T20:15:30.150 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-16T20:15:30.152 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:30.185 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:30 +0000 (0:00:00.072) 0:07:21.599 ******* 2020-11-16T20:15:30.212 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-16T20:15:30.214 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-16T20:15:30.223 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-16T20:15:30.256 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:30 +0000 (0:00:00.071) 0:07:21.671 ******* 2020-11-16T20:15:30.283 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-16T20:15:30.285 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:30.316 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:30 +0000 (0:00:00.059) 0:07:21.730 ******* 2020-11-16T20:15:34.815 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-16T20:15:34.817 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-16T20:15:34.849 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:34 +0000 (0:00:04.533) 0:07:26.264 ******* 2020-11-16T20:15:34.866 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T20:15:34.868 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.010343", "end": "2020-11-16 20:15:34.858343", "rc": 0, "start": "2020-11-16 20:15:34.848000", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-16T20:15:34.901 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:34 +0000 (0:00:00.050) 0:07:26.315 ******* 2020-11-16T20:15:34.928 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2020-11-16T20:15:34.930 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:34.961 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:34 +0000 (0:00:00.060) 0:07:26.375 ******* 2020-11-16T20:15:34.988 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-16T20:15:34.989 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:35.021 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:35 +0000 (0:00:00.060) 0:07:26.436 ******* 2020-11-16T20:15:35.048 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-16T20:15:35.050 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:35.081 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:35 +0000 (0:00:00.059) 0:07:26.495 ******* 2020-11-16T20:15:35.108 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2020-11-16T20:15:35.109 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:35.142 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:35 +0000 (0:00:00.060) 0:07:26.556 ******* 2020-11-16T20:15:35.169 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2020-11-16T20:15:35.171 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:35.203 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:35 +0000 (0:00:00.061) 0:07:26.618 ******* 2020-11-16T20:15:35.230 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2020-11-16T20:15:35.232 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:35.266 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:35 +0000 (0:00:00.062) 0:07:26.680 ******* 2020-11-16T20:15:35.295 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-16T20:15:35.296 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:35.329 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:35 +0000 (0:00:00.063) 0:07:26.744 ******* 2020-11-16T20:15:35.513 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-16T20:15:35.515 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.058396", "end": "2020-11-16 20:15:35.500363", "rc": 0, "start": "2020-11-16 20:15:35.441967", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:15:35.549 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:35 +0000 (0:00:00.219) 0:07:26.963 ******* 2020-11-16T20:15:35.576 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-16T20:15:35.578 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:35.612 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:35 +0000 (0:00:00.063) 0:07:27.027 ******* 2020-11-16T20:15:39.779 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-16T20:15:39.781 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:15:39.818 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:39 +0000 (0:00:04.205) 0:07:31.232 ******* 2020-11-16T20:15:39.948 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-16T20:15:39.950 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.036986", "end": "2020-11-16 20:15:39.936556", "rc": 0, "start": "2020-11-16 20:15:39.899570", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:15:39.984 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:39 +0000 (0:00:00.166) 0:07:31.398 ******* 2020-11-16T20:15:45.828 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-16T20:15:45.830 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605557054, "cache_updated": false, "changed": false} TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi013.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T20:15:45.831 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T20:15:45.832 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-16T20:15:45.867 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:45 +0000 (0:00:05.882) 0:07:37.281 ******* 2020-11-16T20:15:50.102 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:50 +0000 (0:00:04.235) 0:07:41.516 ******* 2020-11-16T20:15:54.307 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T20:15:54.309 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T20:15:54.341 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:54 +0000 (0:00:04.239) 0:07:45.756 ******* 2020-11-16T20:15:55.157 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-16T20:15:55.159 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:15:55.193 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:55 +0000 (0:00:00.852) 0:07:46.608 ******* 2020-11-16T20:15:59.333 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-16T20:15:59.335 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:15:59.367 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:15:59 +0000 (0:00:04.173) 0:07:50.781 ******* 2020-11-16T20:16:03.454 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T20:16:03.456 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} ok: [smithi013.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-16T20:16:03.457 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-16T20:16:03.494 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:03 +0000 (0:00:04.127) 0:07:54.909 ******* 2020-11-16T20:16:03.533 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T20:16:03.534 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-16T20:16:03.566 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:03 +0000 (0:00:00.072) 0:07:54.981 ******* 2020-11-16T20:16:03.593 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T20:16:03.595 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:03.627 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:03 +0000 (0:00:00.060) 0:07:55.042 ******* 2020-11-16T20:16:03.654 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [smithi013.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:16:03.698 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:03 +0000 (0:00:00.070) 0:07:55.112 ******* 2020-11-16T20:16:03.738 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T20:16:03.740 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T20:16:03.750 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T20:16:03.752 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T20:16:03.784 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:03 +0000 (0:00:00.086) 0:07:55.199 ******* 2020-11-16T20:16:03.791 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-16T20:16:03.793 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-16T20:16:03.837 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:03 +0000 (0:00:00.052) 0:07:55.251 ******* 2020-11-16T20:16:03.845 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T20:16:03.847 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:03.882 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:03 +0000 (0:00:00.044) 0:07:55.296 ******* 2020-11-16T20:16:04.726 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-16T20:16:04.727 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:16:04.761 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:04 +0000 (0:00:00.878) 0:07:56.175 ******* 2020-11-16T20:16:12.869 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T20:16:12.871 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:16:12.919 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:12 +0000 (0:00:08.158) 0:08:04.333 ******* 2020-11-16T20:16:25.481 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-16T20:16:25.483 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-16T20:16:33.973 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-16T20:16:34.250 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-16T20:16:34.291 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:34 +0000 (0:00:21.372) 0:08:25.706 ******* 2020-11-16T20:16:34.416 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-16T20:16:34.418 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T20:16:34.451 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:34 +0000 (0:00:00.159) 0:08:25.865 ******* 2020-11-16T20:16:34.775 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-16T20:16:34.777 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-11-16T20:16:35.057 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-16T20:16:35.325 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-16T20:16:35.361 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:35 +0000 (0:00:00.910) 0:08:26.776 ******* 2020-11-16T20:16:35.389 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-16T20:16:35.391 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-16T20:16:35.423 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:35 +0000 (0:00:00.061) 0:08:26.837 ******* 2020-11-16T20:16:35.450 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-16T20:16:35.451 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:35.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:35 +0000 (0:00:00.060) 0:08:26.898 ******* 2020-11-16T20:16:35.511 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-16T20:16:35.513 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:16:35.545 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:35 +0000 (0:00:00.061) 0:08:26.959 ******* 2020-11-16T20:16:35.582 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-16T20:16:35.584 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T20:16:35.594 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T20:16:35.595 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T20:16:35.606 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-16T20:16:35.640 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:35 +0000 (0:00:00.094) 0:08:27.054 ******* 2020-11-16T20:16:35.667 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-16T20:16:35.669 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:35.700 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:35 +0000 (0:00:00.060) 0:08:27.115 ******* 2020-11-16T20:16:36.607 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-16T20:16:36.609 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:16:36.644 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:36 +0000 (0:00:00.944) 0:08:28.059 ******* 2020-11-16T20:16:36.950 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T20:16:36.953 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-16T20:16:36.985 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:36 +0000 (0:00:00.340) 0:08:28.399 ******* 2020-11-16T20:16:41.323 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T20:16:41.324 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-16T20:16:41.360 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:41 +0000 (0:00:04.375) 0:08:32.775 ******* 2020-11-16T20:16:41.505 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T20:16:41.506 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-16T20:16:41.623 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-16T20:16:41.656 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:41 +0000 (0:00:00.296) 0:08:33.071 ******* 2020-11-16T20:16:41.685 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T20:16:41.686 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:41.730 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:41 +0000 (0:00:00.072) 0:08:33.143 ******* 2020-11-16T20:16:41.758 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:16:41.797 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:41 +0000 (0:00:00.067) 0:08:33.211 ******* 2020-11-16T20:16:41.804 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-16T20:16:41.806 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-16T20:16:41.843 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:41 +0000 (0:00:00.046) 0:08:33.257 ******* 2020-11-16T20:16:41.850 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T20:16:41.852 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:41.888 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:41 +0000 (0:00:00.044) 0:08:33.302 ******* 2020-11-16T20:16:42.171 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-16T20:16:42.174 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-16T20:16:42.207 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:42 +0000 (0:00:00.319) 0:08:33.621 ******* 2020-11-16T20:16:42.228 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T20:16:42.230 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-16T20:16:42.266 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:42 +0000 (0:00:00.058) 0:08:33.680 ******* 2020-11-16T20:16:42.370 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-16T20:16:42.372 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-16T20:16:42.477 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-16T20:16:42.513 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:42 +0000 (0:00:00.247) 0:08:33.928 ******* 2020-11-16T20:16:42.552 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-16T20:16:42.554 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:42.588 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:42 +0000 (0:00:00.074) 0:08:34.002 ******* 2020-11-16T20:16:42.628 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** skipping: [smithi013.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-16T20:16:42.662 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:42 +0000 (0:00:00.073) 0:08:34.076 ******* 2020-11-16T20:16:42.664 INFO:teuthology.task.ansible.out: 2020-11-16T20:16:42.690 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-16T20:16:42.692 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:42.727 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:42 +0000 (0:00:00.065) 0:08:34.142 ******* 2020-11-16T20:16:47.054 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-16T20:16:47.055 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 20:07:30 UTC", "ActiveEnterTimestampMonotonic": "21619301", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount basic.target system.slice remote-fs.target network.target -.mount nss-lookup.target systemd-journald.socket sysinit.target systemd-tmpfiles-setup.service time-sync.target local-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 20:07:30 UTC", "AssertTimestampMonotonic": "21617411", "Before": "getty@tty1.service shutdown.target xdm.service plymouth-quit.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 20:07:30 UTC", "ConditionTimestampMonotonic": "21617411", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-16T20:16:47.057 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1151", "ExecMainStartTimestamp": "Mon 2020-11-16 20:07:30 UTC", "ExecMainStartTimestampMonotonic": "21619250", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Mon 2020-11-16 20:07:30 UTC] ; stop_time=[n/a] ; pid=1151 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 20:07:30 UTC", "InactiveExitTimestampMonotonic": "21619301", "InvocationID": "f506e93e75734d93918ffdf746a70ad1", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-11-16T20:16:47.058 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1151", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-11-16T20:16:47.060 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 20:07:30 UTC", "StateChangeTimestampMonotonic": "21619301", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 20:07:30 UTC", "WatchdogTimestampMonotonic": "21619300", "WatchdogUSec": "0"}} 2020-11-16T20:16:47.093 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:04.365) 0:08:38.507 ******* 2020-11-16T20:16:47.111 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-16T20:16:47.112 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-16T20:16:47.146 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.052) 0:08:38.560 ******* 2020-11-16T20:16:47.153 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-16T20:16:47.155 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.192 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.046) 0:08:38.607 ******* 2020-11-16T20:16:47.223 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-16T20:16:47.224 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.257 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.064) 0:08:38.671 ******* 2020-11-16T20:16:47.295 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.334 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.077) 0:08:38.749 ******* 2020-11-16T20:16:47.363 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-11-16T20:16:47.364 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 20:07:17 UTC", "ActiveEnterTimestampMonotonic": "18211592", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target local-fs.target sysinit.target system.slice systemd-tmpfiles-setup.service remote-fs.target systemd-journald.socket basic.target -.mount time-sync.target var-run.mount nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 20:07:17 UTC", "AssertTimestampMonotonic": "18210810", "Before": "getty@tty1.service plymouth-quit.service shutdown.target multi-user.target xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 20:07:17 UTC", "ConditionTimestampMonotonic": "18210810", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-ser 2020-11-16T20:16:47.366 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1234", "ExecMainStartTimestamp": "Mon 2020-11-16 20:07:17 UTC", "ExecMainStartTimestampMonotonic": "18211562", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Mon 2020-11-16 20:07:17 UTC] ; stop_time=[n/a] ; pid=1234 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 20:07:17 UTC", "InactiveExitTimestampMonotonic": "18211592", "InvocationID": "050efe725f6b485fb7c8fa41efe93493", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "i 2020-11-16T20:16:47.369 INFO:teuthology.task.ansible.out:nfinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1234", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", 2020-11-16T20:16:47.370 INFO:teuthology.task.ansible.out: "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 20:07:17 UTC", "StateChangeTimestampMonotonic": "18211592", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 20:07:17 UTC", "WatchdogTimestampMonotonic": "18211591", "WatchdogUSec": "0"}} TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.400 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.065) 0:08:38.814 ******* 2020-11-16T20:16:47.432 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.032) 0:08:38.846 ******* 2020-11-16T20:16:47.440 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-16T20:16:47.442 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.476 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.044) 0:08:38.891 ******* 2020-11-16T20:16:47.485 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-16T20:16:47.492 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.526 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.049) 0:08:38.941 ******* 2020-11-16T20:16:47.533 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-16T20:16:47.535 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.574 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.047) 0:08:38.989 ******* 2020-11-16T20:16:47.581 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-16T20:16:47.583 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.620 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.046) 0:08:39.035 ******* 2020-11-16T20:16:47.628 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.671 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.048) 0:08:39.083 ******* 2020-11-16T20:16:47.676 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.711 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.042) 0:08:39.125 ******* 2020-11-16T20:16:47.719 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.769 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.057) 0:08:39.183 ******* 2020-11-16T20:16:47.776 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-16T20:16:47.778 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.812 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.043) 0:08:39.226 ******* 2020-11-16T20:16:47.819 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.853 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.041) 0:08:39.267 ******* 2020-11-16T20:16:47.862 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.863 INFO:teuthology.task.ansible.out: 2020-11-16T20:16:47.897 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.044) 0:08:39.312 ******* 2020-11-16T20:16:47.903 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-11-16T20:16:47.906 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.942 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.044) 0:08:39.357 ******* 2020-11-16T20:16:47.950 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:47.985 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:47 +0000 (0:00:00.043) 0:08:39.400 ******* 2020-11-16T20:16:48.000 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-16T20:16:48.002 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:48.037 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.052) 0:08:39.452 ******* 2020-11-16T20:16:48.046 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-16T20:16:48.085 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.047) 0:08:39.499 ******* 2020-11-16T20:16:48.092 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:48.137 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.052) 0:08:39.551 ******* 2020-11-16T20:16:48.148 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-16T20:16:48.183 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.046) 0:08:39.598 ******* 2020-11-16T20:16:48.189 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-11-16T20:16:48.191 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:48.229 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.045) 0:08:39.643 ******* 2020-11-16T20:16:48.237 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-16T20:16:48.239 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-16T20:16:48.277 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.048) 0:08:39.692 ******* 2020-11-16T20:16:48.284 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-16T20:16:48.286 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:48.320 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.042) 0:08:39.734 ******* 2020-11-16T20:16:48.329 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T20:16:48.384 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.064) 0:08:39.799 ******* 2020-11-16T20:16:48.392 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-16T20:16:48.394 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-16T20:16:48.434 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.049) 0:08:39.848 ******* 2020-11-16T20:16:48.474 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-16T20:16:48.476 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-16T20:16:48.512 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.078) 0:08:39.927 ******* 2020-11-16T20:16:48.519 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-16T20:16:48.521 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T20:16:48.557 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.044) 0:08:39.972 ******* 2020-11-16T20:16:48.563 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-16T20:16:48.565 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-16T20:16:48.603 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.045) 0:08:40.017 ******* 2020-11-16T20:16:48.646 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-16T20:16:48.648 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-16T20:16:48.685 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.081) 0:08:40.099 ******* 2020-11-16T20:16:48.693 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-16T20:16:48.695 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T20:16:48.730 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.045) 0:08:40.144 ******* 2020-11-16T20:16:48.759 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:48.795 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.065) 0:08:40.209 ******* 2020-11-16T20:16:48.812 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-16T20:16:48.845 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.050) 0:08:40.260 ******* 2020-11-16T20:16:48.852 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-16T20:16:48.854 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:48.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:48 +0000 (0:00:00.043) 0:08:40.303 ******* 2020-11-16T20:16:48.980 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-16T20:16:48.982 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-16T20:16:49.017 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:49 +0000 (0:00:00.128) 0:08:40.432 ******* 2020-11-16T20:16:49.046 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T20:16:49.048 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:49.085 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:49 +0000 (0:00:00.067) 0:08:40.500 ******* 2020-11-16T20:16:49.124 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:49 +0000 (0:00:00.038) 0:08:40.539 ******* 2020-11-16T20:16:49.132 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-16T20:16:49.133 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:16:49.168 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:49 +0000 (0:00:00.043) 0:08:40.583 ******* 2020-11-16T20:16:49.337 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-16T20:16:49.338 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-16T20:16:49.375 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:49 +0000 (0:00:00.206) 0:08:40.789 ******* 2020-11-16T20:16:53.756 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T20:16:53.757 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-16T20:16:53.793 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:53 +0000 (0:00:04.418) 0:08:45.208 ******* 2020-11-16T20:16:53.832 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T20:16:53.833 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi092"}, "changed": false} 2020-11-16T20:16:53.869 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:53 +0000 (0:00:00.075) 0:08:45.284 ******* 2020-11-16T20:16:53.950 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-16T20:16:53.952 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-16T20:16:53.993 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:53 +0000 (0:00:00.123) 0:08:45.407 ******* 2020-11-16T20:16:54.023 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-16T20:16:54.024 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi013"}, "changed": false} 2020-11-16T20:16:54.059 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:16:54 +0000 (0:00:00.066) 0:08:45.473 ******* 2020-11-16T20:17:02.766 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-16T20:17:02.768 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi092.front.sepia.ceph.com", "ansible_hostname": "smithi092", "ansible_nodename": "smithi092"}, "changed": true, "name": "smithi092"} 2020-11-16T20:17:02.799 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:02 +0000 (0:00:08.740) 0:08:54.214 ******* 2020-11-16T20:17:03.115 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-16T20:17:03.117 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-11-16T20:17:03.150 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:03 +0000 (0:00:00.350) 0:08:54.564 ******* 2020-11-16T20:17:03.471 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T20:17:03.473 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-16T20:17:03.504 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:03 +0000 (0:00:00.354) 0:08:54.919 ******* 2020-11-16T20:17:04.376 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-16T20:17:04.378 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T20:17:04.409 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.904) 0:08:55.824 ******* 2020-11-16T20:17:04.436 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-16T20:17:04.437 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:04.469 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.059) 0:08:55.883 ******* 2020-11-16T20:17:04.495 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T20:17:04.497 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:04.530 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.060) 0:08:55.944 ******* 2020-11-16T20:17:04.556 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T20:17:04.558 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:04.589 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.059) 0:08:56.004 ******* 2020-11-16T20:17:04.617 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T20:17:04.618 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:04.650 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.060) 0:08:56.064 ******* 2020-11-16T20:17:04.676 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:04.709 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.058) 0:08:56.123 ******* 2020-11-16T20:17:04.735 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:04.768 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.059) 0:08:56.182 ******* 2020-11-16T20:17:04.795 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:04.828 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.059) 0:08:56.242 ******* 2020-11-16T20:17:04.856 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-16T20:17:04.858 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:04.889 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.061) 0:08:56.303 ******* 2020-11-16T20:17:04.917 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T20:17:04.949 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:04 +0000 (0:00:00.060) 0:08:56.364 ******* 2020-11-16T20:17:04.975 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T20:17:05.008 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.058) 0:08:56.423 ******* 2020-11-16T20:17:05.035 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T20:17:05.067 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.059) 0:08:56.482 ******* 2020-11-16T20:17:05.095 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T20:17:05.096 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.128 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.060) 0:08:56.542 ******* 2020-11-16T20:17:05.166 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T20:17:05.167 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.199 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.071) 0:08:56.614 ******* 2020-11-16T20:17:05.226 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T20:17:05.227 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.261 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.061) 0:08:56.676 ******* 2020-11-16T20:17:05.288 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:17:05.290 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.321 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.059) 0:08:56.735 ******* 2020-11-16T20:17:05.347 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-16T20:17:05.348 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.380 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.059) 0:08:56.794 ******* 2020-11-16T20:17:05.408 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.443 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.062) 0:08:56.857 ******* 2020-11-16T20:17:05.471 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T20:17:05.472 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.506 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.063) 0:08:56.920 ******* 2020-11-16T20:17:05.533 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-16T20:17:05.534 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.565 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.059) 0:08:56.980 ******* 2020-11-16T20:17:05.592 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T20:17:05.593 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.624 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.059) 0:08:57.039 ******* 2020-11-16T20:17:05.658 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T20:17:05.659 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.691 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.066) 0:08:57.105 ******* 2020-11-16T20:17:05.717 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T20:17:05.719 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.750 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.059) 0:08:57.165 ******* 2020-11-16T20:17:05.778 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T20:17:05.779 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.810 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.059) 0:08:57.225 ******* 2020-11-16T20:17:05.836 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T20:17:05.838 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.869 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.058) 0:08:57.283 ******* 2020-11-16T20:17:05.895 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T20:17:05.897 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.928 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.059) 0:08:57.342 ******* 2020-11-16T20:17:05.954 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-16T20:17:05.955 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:05.989 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:05 +0000 (0:00:00.061) 0:08:57.404 ******* 2020-11-16T20:17:06.015 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-16T20:17:06.017 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:06.048 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.058) 0:08:57.462 ******* 2020-11-16T20:17:06.076 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T20:17:06.077 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:06.108 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.060) 0:08:57.523 ******* 2020-11-16T20:17:06.134 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:06.167 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.058) 0:08:57.581 ******* 2020-11-16T20:17:06.193 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-16T20:17:06.195 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:06.227 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.059) 0:08:57.641 ******* 2020-11-16T20:17:06.254 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T20:17:06.255 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:06.287 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.060) 0:08:57.701 ******* 2020-11-16T20:17:06.314 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-16T20:17:06.316 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:06.347 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.060) 0:08:57.762 ******* 2020-11-16T20:17:06.374 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:06.407 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.059) 0:08:57.822 ******* 2020-11-16T20:17:06.538 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-16T20:17:06.540 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.003337", "end": "2020-11-16 20:17:06.527119", "rc": 0, "start": "2020-11-16 20:17:06.523782", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:17:06.572 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.164) 0:08:57.987 ******* 2020-11-16T20:17:06.599 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T20:17:06.635 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.062) 0:08:58.049 ******* 2020-11-16T20:17:06.799 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-16T20:17:06.801 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi013.front.sepia.ceph.com", "ansible_hostname": "smithi013", "ansible_nodename": "smithi013"}, "changed": true, "name": "smithi013"} 2020-11-16T20:17:06.835 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:06 +0000 (0:00:00.199) 0:08:58.249 ******* 2020-11-16T20:17:06.968 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T20:17:06.969 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-16T20:17:07.010 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:07 +0000 (0:00:00.174) 0:08:58.424 ******* 2020-11-16T20:17:07.037 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T20:17:07.039 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:07.072 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:07 +0000 (0:00:00.062) 0:08:58.486 ******* 2020-11-16T20:17:07.151 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2020-11-16T20:17:07.153 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-11-16T20:17:07.189 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:07 +0000 (0:00:00.117) 0:08:58.604 ******* 2020-11-16T20:17:07.526 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-11-16T20:17:07.528 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-16T20:17:07.564 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:07 +0000 (0:00:00.374) 0:08:58.979 ******* 2020-11-16T20:17:08.079 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** ok: [smithi013.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-16T20:17:08.115 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.550) 0:08:59.530 ******* 2020-11-16T20:17:08.145 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.179 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.063) 0:08:59.593 ******* 2020-11-16T20:17:08.206 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T20:17:08.209 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.240 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.061) 0:08:59.654 ******* 2020-11-16T20:17:08.267 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T20:17:08.269 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.301 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.061) 0:08:59.715 ******* 2020-11-16T20:17:08.329 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.365 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.064) 0:08:59.780 ******* 2020-11-16T20:17:08.393 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T20:17:08.394 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.426 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.060) 0:08:59.841 ******* 2020-11-16T20:17:08.453 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T20:17:08.455 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.490 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.063) 0:08:59.904 ******* 2020-11-16T20:17:08.497 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T20:17:08.499 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-16T20:17:08.520 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.553 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.063) 0:08:59.968 ******* 2020-11-16T20:17:08.581 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-16T20:17:08.583 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.616 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.062) 0:09:00.030 ******* 2020-11-16T20:17:08.644 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-16T20:17:08.680 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.063) 0:09:00.094 ******* 2020-11-16T20:17:08.707 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-16T20:17:08.741 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.061) 0:09:00.156 ******* 2020-11-16T20:17:08.769 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-16T20:17:08.804 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.062) 0:09:00.218 ******* 2020-11-16T20:17:08.832 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-16T20:17:08.833 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.865 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.061) 0:09:00.280 ******* 2020-11-16T20:17:08.905 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-16T20:17:08.906 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-16T20:17:08.944 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:08 +0000 (0:00:00.078) 0:09:00.359 ******* 2020-11-16T20:17:08.972 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-16T20:17:08.974 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:09.007 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:09 +0000 (0:00:00.063) 0:09:00.422 ******* 2020-11-16T20:17:09.036 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:17:09.037 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:09.069 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:09 +0000 (0:00:00.061) 0:09:00.483 ******* 2020-11-16T20:17:09.096 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-16T20:17:09.098 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:09.129 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:09 +0000 (0:00:00.060) 0:09:00.544 ******* 2020-11-16T20:17:09.157 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-16T20:17:09.158 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:09.190 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:09 +0000 (0:00:00.060) 0:09:00.605 ******* 2020-11-16T20:17:09.218 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-16T20:17:09.220 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:09.254 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:09 +0000 (0:00:00.063) 0:09:00.669 ******* 2020-11-16T20:17:09.281 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-16T20:17:09.283 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:09.315 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:09 +0000 (0:00:00.060) 0:09:00.729 ******* 2020-11-16T20:17:13.363 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T20:17:13.364 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-16T20:17:13.397 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:04.082) 0:09:04.812 ******* 2020-11-16T20:17:13.403 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T20:17:13.405 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.438 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.040) 0:09:04.853 ******* 2020-11-16T20:17:13.466 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T20:17:13.468 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.499 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.060) 0:09:04.913 ******* 2020-11-16T20:17:13.526 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T20:17:13.527 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.558 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.059) 0:09:04.973 ******* 2020-11-16T20:17:13.586 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-16T20:17:13.587 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.619 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.060) 0:09:05.033 ******* 2020-11-16T20:17:13.647 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T20:17:13.648 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.679 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.060) 0:09:05.094 ******* 2020-11-16T20:17:13.711 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T20:17:13.713 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.744 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.064) 0:09:05.159 ******* 2020-11-16T20:17:13.772 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-16T20:17:13.774 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.805 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.060) 0:09:05.219 ******* 2020-11-16T20:17:13.833 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.866 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.060) 0:09:05.280 ******* 2020-11-16T20:17:13.892 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-16T20:17:13.894 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.926 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.059) 0:09:05.340 ******* 2020-11-16T20:17:13.953 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-16T20:17:13.955 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:13.986 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:13 +0000 (0:00:00.060) 0:09:05.401 ******* 2020-11-16T20:17:14.018 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:14.052 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.065) 0:09:05.467 ******* 2020-11-16T20:17:14.080 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-16T20:17:14.082 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:14.116 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.063) 0:09:05.531 ******* 2020-11-16T20:17:14.146 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:14.180 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.063) 0:09:05.594 ******* 2020-11-16T20:17:14.208 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:14.241 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.061) 0:09:05.656 ******* 2020-11-16T20:17:14.259 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T20:17:14.261 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:17:14.297 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.055) 0:09:05.711 ******* 2020-11-16T20:17:14.327 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T20:17:14.363 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.066) 0:09:05.778 ******* 2020-11-16T20:17:14.383 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-16T20:17:14.384 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005252", "end": "2020-11-16 20:17:14.366805", "rc": 0, "start": "2020-11-16 20:17:14.361553", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:17:14.433 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.070) 0:09:05.848 ******* 2020-11-16T20:17:14.463 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-16T20:17:14.500 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.066) 0:09:05.914 ******* 2020-11-16T20:17:14.520 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T20:17:14.522 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.029875", "end": "2020-11-16 20:17:14.505185", "rc": 0, "start": "2020-11-16 20:17:14.475310", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:17:14.557 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:14 +0000 (0:00:00.056) 0:09:05.971 ******* 2020-11-16T20:17:18.915 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-11-16T20:17:18.918 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-16T20:17:18.955 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:18 +0000 (0:00:04.397) 0:09:10.369 ******* 2020-11-16T20:17:18.983 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-16T20:17:18.985 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:19.022 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:19 +0000 (0:00:00.067) 0:09:10.437 ******* 2020-11-16T20:17:20.077 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-11-16T20:17:20.079 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-16T20:17:21.031 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-16T20:17:21.065 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:21 +0000 (0:00:02.042) 0:09:12.479 ******* 2020-11-16T20:17:21.561 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T20:17:21.563 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:17:21.596 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:21 +0000 (0:00:00.531) 0:09:13.010 ******* 2020-11-16T20:17:30.042 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-16T20:17:30.044 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:17:30.077 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:30 +0000 (0:00:08.480) 0:09:21.491 ******* 2020-11-16T20:17:30.104 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-16T20:17:30.138 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:30 +0000 (0:00:00.061) 0:09:21.553 ******* 2020-11-16T20:17:30.311 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-11-16T20:17:30.313 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.035885", "end": "2020-11-16 20:17:30.288491", "rc": 0, "start": "2020-11-16 20:17:30.252606", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:17:30.348 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:30 +0000 (0:00:00.209) 0:09:21.763 ******* 2020-11-16T20:17:33.254 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-16T20:17:33.256 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:17:33.287 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:33 +0000 (0:00:02.938) 0:09:24.701 ******* 2020-11-16T20:17:38.769 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-16T20:17:38.770 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 1 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137606 files and directories 2020-11-16T20:17:38.772 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 1 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-16T20:17:38.773 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137606 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-16T20:17:38.806 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:38 +0000 (0:00:05.519) 0:09:30.221 ******* 2020-11-16T20:17:43.759 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T20:17:43.761 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:17:43.799 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:43 +0000 (0:00:04.992) 0:09:35.214 ******* 2020-11-16T20:17:43.827 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T20:17:43.829 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:43.860 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:43 +0000 (0:00:00.061) 0:09:35.275 ******* 2020-11-16T20:17:46.550 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-16T20:17:46.552 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 1 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137606 files and directories 2020-11-16T20:17:46.554 INFO:teuthology.task.ansible.out: currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 1 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Readin 2020-11-16T20:17:46.555 INFO:teuthology.task.ansible.out:g database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137606 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-16T20:17:46.585 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:46 +0000 (0:00:02.724) 0:09:38.000 ******* 2020-11-16T20:17:54.660 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T20:17:54.662 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 1 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9656 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-16T20:17:54.664 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137464 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-16T20:17:54.665 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 1 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9656 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-16T20:17:54.667 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137464 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-16T20:17:54.696 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:54 +0000 (0:00:08.111) 0:09:46.111 ******* 2020-11-16T20:17:55.565 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T20:17:55.567 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:17:55.600 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:55 +0000 (0:00:00.903) 0:09:47.014 ******* 2020-11-16T20:17:55.608 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-16T20:17:55.609 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:17:55.642 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:55 +0000 (0:00:00.042) 0:09:47.057 ******* 2020-11-16T20:17:55.649 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-16T20:17:55.650 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:55.686 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:55 +0000 (0:00:00.043) 0:09:47.100 ******* 2020-11-16T20:17:55.764 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T20:17:55.766 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-16T20:17:55.800 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:55 +0000 (0:00:00.114) 0:09:47.214 ******* 2020-11-16T20:17:55.827 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-16T20:17:55.829 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:17:55.860 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:17:55 +0000 (0:00:00.060) 0:09:47.275 ******* 2020-11-16T20:18:12.045 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T20:18:12.046 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 1 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (8579 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 2020-11-16T20:18:12.048 INFO:teuthology.task.ansible.out: 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 137464 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu 2020-11-16T20:18:12.049 INFO:teuthology.task.ansible.out:1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 1 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.4 [2727 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.4 [924 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (8579 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Rea 2020-11-16T20:18:12.051 INFO:teuthology.task.ansible.out:ding database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 137464 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.4_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-16T20:18:12.086 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:12 +0000 (0:00:16.225) 0:10:03.500 ******* 2020-11-16T20:18:12.177 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-16T20:18:12.179 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-16T20:18:12.214 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:12 +0000 (0:00:00.128) 0:10:03.629 ******* 2020-11-16T20:18:12.526 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-16T20:18:12.528 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-16T20:18:12.560 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:12 +0000 (0:00:00.346) 0:10:03.975 ******* 2020-11-16T20:18:12.715 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T20:18:12.717 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T20:18:12.750 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:12 +0000 (0:00:00.189) 0:10:04.164 ******* 2020-11-16T20:18:12.883 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T20:18:12.923 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:12 +0000 (0:00:00.173) 0:10:04.337 ******* 2020-11-16T20:18:13.002 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-16T20:18:13.004 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:18:13.037 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:13 +0000 (0:00:00.114) 0:10:04.452 ******* 2020-11-16T20:18:21.284 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T20:18:21.286 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 20:07:30 UTC", "ActiveEnterTimestampMonotonic": "22025122", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target remote-fs.target systemd-journald.socket systemd-tmpfiles-setup.service basic.target nss-lookup.target sysinit.target system.slice -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 20:07:30 UTC", "AssertTimestampMonotonic": "21612966", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 20:07:30 UTC", "ConditionTimestampMonotonic": "21612966", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-16T20:18:21.287 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1205", "ExecMainStartTimestamp": "Mon 2020-11-16 20:07:30 UTC", "ExecMainStartTimestampMonotonic": "22025093", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 20:07:30 UTC", "InactiveExitTimestampMonotonic": "21614606", "InvocationID": "fbb24c26825a4f3fa22c1e4434d0769b", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-16T20:18:21.289 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1205", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-16T20:18:21.290 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 20:07:30 UTC", "StateChangeTimestampMonotonic": "22025122", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 20:07:30 UTC", "WatchdogTimestampMonotonic": "22025120", "WatchdogUSec": "0"}} 2020-11-16T20:18:21.331 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:21 +0000 (0:00:08.294) 0:10:12.746 ******* 2020-11-16T20:18:21.359 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T20:18:21.361 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:21.395 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:21 +0000 (0:00:00.063) 0:10:12.809 ******* 2020-11-16T20:18:25.411 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-16T20:18:25.413 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-16T20:18:25.446 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:25 +0000 (0:00:04.051) 0:10:16.860 ******* 2020-11-16T20:18:25.453 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T20:18:25.454 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:25.487 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:25 +0000 (0:00:00.041) 0:10:16.901 ******* 2020-11-16T20:18:25.495 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-16T20:18:25.496 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:25.527 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:25 +0000 (0:00:00.040) 0:10:16.942 ******* 2020-11-16T20:18:25.534 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T20:18:25.536 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:25.572 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:25 +0000 (0:00:00.044) 0:10:16.987 ******* 2020-11-16T20:18:38.083 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:38.118 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:38 +0000 (0:00:12.545) 0:10:29.532 ******* 2020-11-16T20:18:38.145 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T20:18:38.147 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:38.178 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:38 +0000 (0:00:00.060) 0:10:29.593 ******* 2020-11-16T20:18:38.185 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-16T20:18:38.186 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-16T20:18:38.226 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:38 +0000 (0:00:00.042) 0:10:29.635 ******* 2020-11-16T20:18:38.234 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T20:18:38.236 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:38.269 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:38 +0000 (0:00:00.048) 0:10:29.683 ******* 2020-11-16T20:18:38.296 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T20:18:38.298 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:38.330 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:38 +0000 (0:00:00.061) 0:10:29.745 ******* 2020-11-16T20:18:38.357 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T20:18:38.359 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:38.393 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:38 +0000 (0:00:00.062) 0:10:29.807 ******* 2020-11-16T20:18:38.420 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:18:38.422 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:38.456 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:38 +0000 (0:00:00.063) 0:10:29.871 ******* 2020-11-16T20:18:38.483 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T20:18:38.484 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:38.518 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:38 +0000 (0:00:00.061) 0:10:29.933 ******* 2020-11-16T20:18:42.544 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-16T20:18:42.546 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-16T20:18:42.578 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:04.060) 0:10:33.993 ******* 2020-11-16T20:18:42.586 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T20:18:42.588 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:42.622 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:00.043) 0:10:34.036 ******* 2020-11-16T20:18:42.650 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T20:18:42.661 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:42.685 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:00.062) 0:10:34.099 ******* 2020-11-16T20:18:42.712 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T20:18:42.714 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:42.746 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:00.061) 0:10:34.160 ******* 2020-11-16T20:18:42.753 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-16T20:18:42.754 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-16T20:18:42.791 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:00.045) 0:10:34.206 ******* 2020-11-16T20:18:42.799 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T20:18:42.801 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:42.835 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:00.043) 0:10:34.249 ******* 2020-11-16T20:18:42.862 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T20:18:42.864 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:42.896 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:00.061) 0:10:34.311 ******* 2020-11-16T20:18:42.924 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T20:18:42.925 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [smithi013.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T20:18:42.958 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:00.061) 0:10:34.373 ******* 2020-11-16T20:18:42.993 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:42 +0000 (0:00:00.034) 0:10:34.407 ******* 2020-11-16T20:18:43.019 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T20:18:43.021 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.053 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.060) 0:10:34.468 ******* 2020-11-16T20:18:43.080 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T20:18:43.082 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.114 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.061) 0:10:34.529 ******* 2020-11-16T20:18:43.141 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T20:18:43.143 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.175 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.060) 0:10:34.589 ******* 2020-11-16T20:18:43.202 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T20:18:43.203 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.240 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.065) 0:10:34.655 ******* 2020-11-16T20:18:43.257 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-16T20:18:43.258 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 20:07:18 UTC", "ActiveEnterTimestampMonotonic": "18621855", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target nss-lookup.target systemd-journald.socket -.mount systemd-tmpfiles-setup.service sysinit.target network.target system.slice remote-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 20:07:17 UTC", "AssertTimestampMonotonic": "18213397", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 20:07:17 UTC", "ConditionTimestampMonotonic": "18213396", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2020-11-16T20:18:43.260 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1331", "ExecMainStartTimestamp": "Mon 2020-11-16 20:07:18 UTC", "ExecMainStartTimestampMonotonic": "18621842", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 20:07:17 UTC", "InactiveExitTimestampMonotonic": "18214398", "InvocationID": "4259e4691a5d406b9e109bbb2809cb8c", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2020-11-16T20:18:43.261 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1331", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2020-11-16T20:18:43.263 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 20:07:18 UTC", "StateChangeTimestampMonotonic": "18621855", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 20:07:18 UTC", "WatchdogTimestampMonotonic": "18621854", "WatchdogUSec": "0"}} 2020-11-16T20:18:43.292 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.051) 0:10:34.706 ******* 2020-11-16T20:18:43.299 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T20:18:43.300 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.332 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.040) 0:10:34.747 ******* 2020-11-16T20:18:43.338 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-16T20:18:43.340 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.373 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.041) 0:10:34.788 ******* 2020-11-16T20:18:43.380 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T20:18:43.382 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.415 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.041) 0:10:34.830 ******* 2020-11-16T20:18:43.422 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-16T20:18:43.424 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.460 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.044) 0:10:34.875 ******* 2020-11-16T20:18:43.467 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T20:18:43.469 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.502 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.041) 0:10:34.916 ******* 2020-11-16T20:18:43.516 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-16T20:18:43.517 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.550 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.048) 0:10:34.964 ******* 2020-11-16T20:18:43.557 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T20:18:43.559 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.593 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.042) 0:10:35.007 ******* 2020-11-16T20:18:43.600 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-16T20:18:43.601 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.635 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.042) 0:10:35.050 ******* 2020-11-16T20:18:43.645 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T20:18:43.647 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.693 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.057) 0:10:35.108 ******* 2020-11-16T20:18:43.700 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-16T20:18:43.702 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.743 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.050) 0:10:35.158 ******* 2020-11-16T20:18:43.750 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-16T20:18:43.752 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.785 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.041) 0:10:35.200 ******* 2020-11-16T20:18:43.794 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-16T20:18:43.796 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:43.836 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:43 +0000 (0:00:00.050) 0:10:35.251 ******* 2020-11-16T20:18:47.843 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T20:18:47.845 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-16T20:18:47.878 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:47 +0000 (0:00:04.041) 0:10:39.292 ******* 2020-11-16T20:18:47.885 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-16T20:18:47.886 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:47.921 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:47 +0000 (0:00:00.042) 0:10:39.335 ******* 2020-11-16T20:18:47.948 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-16T20:18:47.949 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:47.982 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:47 +0000 (0:00:00.060) 0:10:39.396 ******* 2020-11-16T20:18:48.009 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.042 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.060) 0:10:39.456 ******* 2020-11-16T20:18:48.068 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-16T20:18:48.070 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.102 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.060) 0:10:39.516 ******* 2020-11-16T20:18:48.129 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-16T20:18:48.130 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.169 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.067) 0:10:39.584 ******* 2020-11-16T20:18:48.187 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T20:18:48.188 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-16T20:18:48.221 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.051) 0:10:39.635 ******* 2020-11-16T20:18:48.228 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-16T20:18:48.229 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.268 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.047) 0:10:39.683 ******* 2020-11-16T20:18:48.276 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T20:18:48.277 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.311 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.042) 0:10:39.725 ******* 2020-11-16T20:18:48.316 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T20:18:48.318 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.354 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.043) 0:10:39.769 ******* 2020-11-16T20:18:48.381 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T20:18:48.383 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.415 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.061) 0:10:39.830 ******* 2020-11-16T20:18:48.442 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T20:18:48.444 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.476 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.060) 0:10:39.890 ******* 2020-11-16T20:18:48.502 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T20:18:48.504 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.547 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.071) 0:10:39.961 ******* 2020-11-16T20:18:48.574 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T20:18:48.575 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.607 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.060) 0:10:40.022 ******* 2020-11-16T20:18:48.634 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T20:18:48.636 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload weekly kernel-clean crontab.] ************************** ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-16T20:18:48.668 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.061) 0:10:40.083 ******* 2020-11-16T20:18:48.699 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.030) 0:10:40.114 ******* 2020-11-16T20:18:48.709 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-16T20:18:48.711 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.748 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.048) 0:10:40.162 ******* 2020-11-16T20:18:48.776 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T20:18:48.778 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.809 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.061) 0:10:40.224 ******* 2020-11-16T20:18:48.839 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-16T20:18:48.841 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.872 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.062) 0:10:40.286 ******* 2020-11-16T20:18:48.905 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.937 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.065) 0:10:40.352 ******* 2020-11-16T20:18:48.965 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-16T20:18:48.967 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:48.998 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:48 +0000 (0:00:00.060) 0:10:40.413 ******* 2020-11-16T20:18:49.026 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-16T20:18:49.028 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:49.060 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:49 +0000 (0:00:00.061) 0:10:40.474 ******* 2020-11-16T20:18:49.092 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:49 +0000 (0:00:00.031) 0:10:40.506 ******* 2020-11-16T20:18:49.099 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-16T20:18:49.101 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:49.135 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:49 +0000 (0:00:00.043) 0:10:40.550 ******* 2020-11-16T20:18:49.167 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:49.206 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:49 +0000 (0:00:00.070) 0:10:40.620 ******* 2020-11-16T20:18:49.348 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-11-16T20:18:49.350 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-16T20:18:49.382 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:49 +0000 (0:00:00.176) 0:10:40.796 ******* 2020-11-16T20:18:53.495 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-16T20:18:53.497 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-16T20:18:53.531 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:53 +0000 (0:00:04.148) 0:10:44.946 ******* 2020-11-16T20:18:53.663 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T20:18:53.665 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T20:18:53.697 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:53 +0000 (0:00:00.165) 0:10:45.111 ******* 2020-11-16T20:18:53.723 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T20:18:53.725 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:53.757 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:53 +0000 (0:00:00.059) 0:10:45.171 ******* 2020-11-16T20:18:53.774 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-16T20:18:53.775 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-16T20:18:53.807 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:53 +0000 (0:00:00.050) 0:10:45.222 ******* 2020-11-16T20:18:53.814 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-16T20:18:53.816 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:53.848 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:53 +0000 (0:00:00.040) 0:10:45.262 ******* 2020-11-16T20:18:53.861 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-16T20:18:53.863 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:53.894 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:53 +0000 (0:00:00.046) 0:10:45.309 ******* 2020-11-16T20:18:53.900 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-16T20:18:53.902 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:53.935 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:53 +0000 (0:00:00.040) 0:10:45.350 ******* 2020-11-16T20:18:53.962 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-16T20:18:53.964 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:53.998 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:53 +0000 (0:00:00.062) 0:10:45.412 ******* 2020-11-16T20:18:54.025 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-16T20:18:54.027 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.059 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.061) 0:10:45.474 ******* 2020-11-16T20:18:54.086 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:18:54.088 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-16T20:18:54.122 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.062) 0:10:45.536 ******* 2020-11-16T20:18:54.180 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T20:18:54.182 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.192 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.213 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.215 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} TASK [testnode : Upload weekly kernel-clean crontab.] ************************** ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-16T20:18:54.261 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.138) 0:10:45.674 ******* 2020-11-16T20:18:54.270 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T20:18:54.271 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-16T20:18:54.272 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.274 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.285 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.296 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.300 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.308 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.352 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.091) 0:10:45.766 ******* 2020-11-16T20:18:54.565 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi092.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004251", "end": "2020-11-16 20:18:54.532577", "rc": 0, "start": "2020-11-16 20:18:54.528326", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T20:18:54.603 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.251) 0:10:46.018 ******* 2020-11-16T20:18:54.609 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-16T20:18:54.611 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-16T20:18:54.651 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.048) 0:10:46.066 ******* 2020-11-16T20:18:54.659 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-16T20:18:54.660 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.705 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.053) 0:10:46.119 ******* 2020-11-16T20:18:54.744 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-16T20:18:54.746 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.780 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.075) 0:10:46.195 ******* 2020-11-16T20:18:54.819 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-16T20:18:54.821 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:54.860 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:54 +0000 (0:00:00.078) 0:10:46.273 ******* 2020-11-16T20:18:54.974 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-16T20:18:54.975 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-16T20:18:55.011 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:55 +0000 (0:00:00.151) 0:10:46.425 ******* 2020-11-16T20:18:55.723 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T20:18:55.725 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:18:55.758 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:55 +0000 (0:00:00.747) 0:10:47.172 ******* 2020-11-16T20:18:59.829 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-16T20:18:59.831 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T20:18:59.863 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:59 +0000 (0:00:04.105) 0:10:51.278 ******* 2020-11-16T20:18:59.870 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-16T20:18:59.872 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:59.906 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:59 +0000 (0:00:00.042) 0:10:51.321 ******* 2020-11-16T20:18:59.913 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-16T20:18:59.914 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:18:59.947 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:59 +0000 (0:00:00.041) 0:10:51.362 ******* 2020-11-16T20:18:59.953 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T20:18:59.955 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'block_used': 13753462, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341646, 'size_available': 927095382016, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149530, 'block_size': 4096, 'inode_available': 60905446}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341646, "block_size": 4096, "block_total": 240095108, "block_used": 13753462, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905446, "inode_total": 61054976, "inode_used": 149530, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927095382016, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-16T20:18:59.987 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:18:59 +0000 (0:00:00.039) 0:10:51.402 ******* 2020-11-16T20:18:59.993 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-16T20:18:59.994 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.028 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.040) 0:10:51.442 ******* 2020-11-16T20:19:00.055 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2020-11-16T20:19:00.057 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.089 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.060) 0:10:51.503 ******* 2020-11-16T20:19:00.115 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-16T20:19:00.116 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010765", "end": "2020-11-16 20:19:00.102854", "rc": 0, "start": "2020-11-16 20:19:00.092089", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:00.149 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.060) 0:10:51.564 ******* 2020-11-16T20:19:00.184 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.034) 0:10:51.598 ******* 2020-11-16T20:19:00.191 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-16T20:19:00.192 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.226 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.042) 0:10:51.641 ******* 2020-11-16T20:19:00.233 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-16T20:19:00.234 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'block_used': 13753462, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341646, 'size_available': 927095382016, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149530, 'block_size': 4096, 'inode_available': 60905446}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341646, "block_size": 4096, "block_total": 240095108, "block_used": 13753462, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905446, "inode_total": 61054976, "inode_used": 149530, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927095382016, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.267 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.041) 0:10:51.682 ******* 2020-11-16T20:19:00.273 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:19:00.275 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-16T20:19:00.308 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.040) 0:10:51.722 ******* 2020-11-16T20:19:00.315 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T20:19:00.349 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.041) 0:10:51.764 ******* 2020-11-16T20:19:00.356 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T20:19:00.358 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.368 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.370 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.380 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.381 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.391 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-16T20:19:00.392 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.402 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-16T20:19:00.404 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.413 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.415 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.425 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.435 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.437 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.446 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-16T20:19:00.486 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:00 +0000 (0:00:00.136) 0:10:51.901 ******* 2020-11-16T20:19:01.660 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T20:19:01.662 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAc1ROTDuEMXPdJSJUWhk7fn09eKdxgQcZ'], 'uuids': ['acd81826-3709-43bc-a3a7-64f04664ccbd']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.133645", "end": "2020-11-16 20:19:01.629640", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAc1ROTDuEMXPdJSJUWhk7fn09eKdxgQcZ"], "labels": [], "masters": [], "uuids": ["acd81826-3709-43bc-a3a7-64f04664ccbd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:00.495995", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:01.674 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:01.675 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:07.098 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQACn1mF0QzHryzG8l5I80odwwscXhi6cIn'], 'uuids': ['a87edfac-b925-4593-af28-e0146bfa7ceb']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.105472", "end": "2020-11-16 20:19:07.036177", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQACn1mF0QzHryzG8l5I80odwwscXhi6cIn"], "labels": [], "masters": [], "uuids": ["a87edfac-b925-4593-af28-e0146bfa7ceb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:05.930705", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:07.099 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5LJ4S', 'wwn-0x5000c500918da5aa'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5LJ4S-part1', 'wwn-0x5000c500918da5aa-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500918da5aa', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5LJ4S', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5LJ4S", "wwn-0x5000c500918da5aa"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5LJ4S-part1", "wwn-0x5000c500918da5aa-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5LJ4S", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918da5aa"}}, "skip_reason": 2020-11-16T20:19:07.101 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-16T20:19:08.876 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-16T20:19:08.878 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:08.208935", "end": "2020-11-16 20:19:08.857239", "rc": 0, "start": "2020-11-16 20:19:00.648304", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-16T20:19:08.913 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:08 +0000 (0:00:08.426) 0:11:00.328 ******* 2020-11-16T20:19:08.939 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2020-11-16T20:19:08.941 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:08.977 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:08 +0000 (0:00:00.064) 0:11:00.392 ******* 2020-11-16T20:19:09.007 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-16T20:19:09.009 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:09.041 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:09 +0000 (0:00:00.063) 0:11:00.455 ******* 2020-11-16T20:19:09.068 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-16T20:19:09.070 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:09.102 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:09 +0000 (0:00:00.060) 0:11:00.516 ******* 2020-11-16T20:19:12.389 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T20:19:12.392 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-zd03GV-PIJV-66Au-JSKy-O87e-3lmW-4phd4a', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400QM400BGN', 'nvme-nvme.8086-50484654363230343030514d34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.160788", "end": "2020-11-16 20:19:12.364244", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-zd03GV-PIJV-66Au-JSKy-O87e-3lmW-4phd4a", "nvme-INTEL_SSDPEDMD400G4_PHFT620400QM400BGN", "nvme-nvme.8086-50484654363230343030514d34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:11.203456", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-16T20:19:12.393 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:12.402 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:12.413 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:13.686 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAvQHkij04f2RVR3bx5wx87YwINetsuAMS'], 'uuids': ['aa638380-67c6-41a2-aeeb-be24598c3a77']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.155170", "end": "2020-11-16 20:19:13.660699", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAvQHkij04f2RVR3bx5wx87YwINetsuAMS"], "labels": [], "masters": [], "uuids": ["aa638380-67c6-41a2-aeeb-be24598c3a77"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:12.505529", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:13.697 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:13.709 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:18.166 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-16T20:19:18.167 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:19:18.199 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:18 +0000 (0:00:09.097) 0:11:09.614 ******* 2020-11-16T20:19:18.227 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-11-16T20:19:18.228 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:18.261 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:18 +0000 (0:00:00.061) 0:11:09.675 ******* 2020-11-16T20:19:18.303 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T20:19:18.305 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item={'block_used': 13753475, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341633, 'size_available': 927095328768, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149529, 'block_size': 4096, 'inode_available': 60905447}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341633, "block_size": 4096, "block_total": 240095108, "block_used": 13753475, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905447, "inode_total": 61054976, "inode_used": 149529, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927095328768, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-16T20:19:18.337 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:18 +0000 (0:00:00.076) 0:11:09.751 ******* 2020-11-16T20:19:23.175 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-16T20:19:23.177 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011272", "end": "2020-11-16 20:19:22.628924", "rc": 0, "start": "2020-11-16 20:19:22.617652", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:23.210 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:23 +0000 (0:00:04.872) 0:11:14.624 ******* 2020-11-16T20:19:23.248 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-16T20:19:23.249 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'block_used': 13753475, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341633, 'size_available': 927095328768, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149529, 'block_size': 4096, 'inode_available': 60905447}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341633, "block_size": 4096, "block_total": 240095108, "block_used": 13753475, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905447, "inode_total": 61054976, "inode_used": 149529, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927095328768, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T20:19:23.250 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQALQ5OQiHw1Lmsead3YUN98WhBeNoLz3wB'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.093367", "end": "2020-11-16 20:19:19.060896", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQALQ5OQiHw1Lmsead3YUN98WhBeNoLz3wB"], "labels": [], "masters": [], "uuids": []}, "model": null, "parti 2020-11-16T20:19:23.251 INFO:teuthology.task.ansible.out:tions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:17.967529", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:23.287 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:23 +0000 (0:00:00.076) 0:11:14.701 ******* 2020-11-16T20:19:23.296 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:23.318 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-16T20:19:23.369 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:23 +0000 (0:00:00.082) 0:11:14.784 ******* 2020-11-16T20:19:23.411 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T20:19:23.412 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:28.147 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAvlbeWMJDJe5uoVT0MCfcJmopYlRVbWSf'], 'uuids': ['ac498e8e-e990-472e-aa27-272f5958531a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.125017", "end": "2020-11-16 20:19:24.476687", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAvlbeWMJDJe5uoVT0MCfcJmopYlRVbWSf"], "labels": [], "masters": [], "uuids": ["ac498e8e-e990-472e-aa27-272f5958531a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:23.351670", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:28.162 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHt0arNYU9dY8S9G88CqXe1QcEiu032Ceyv'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.155579", "end": "2020-11-16 20:19:24.675694", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHt0arNYU9dY8S9G88CqXe1QcEiu032Ceyv"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:23.520115", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:28.163 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:28.164 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:28.165 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtCJPFJ1r3PlLKH2K4UBvzrZMY6l2HpSTZ'], 'uuids': ['c242352a-d5c3-4f3d-8bb8-ceb7bb191852']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.251667", "end": "2020-11-16 20:19:26.068141", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtCJPFJ1r3PlLKH2K4UBvzrZMY6l2HpSTZ"], "labels": [], "masters": [], "uuids": ["c242352a-d5c3-4f3d-8bb8-ceb7bb191852"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", " 2020-11-16T20:19:28.166 INFO:teuthology.task.ansible.out:sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:24.816474", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2YD', 'wwn-0x5000c500871b846a'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2YD-part1', 'wwn-0x5000c500871b846a-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b846a', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J2YD', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C 2020-11-16T20:19:28.168 INFO:teuthology.task.ansible.out:610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2YD", "wwn-0x5000c500871b846a"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2YD-part1", "wwn-0x5000c500871b846a-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2YD", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b846a"}}, "skip_reason": "Conditional result was False"} changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-sfvFp0-2hQ3-QH8y-3ale-Bo6L-TzSU-RdAfDN', 'nvme-INTEL_SSDPED1D480GA_PHMB7476008K480DGN', 'nvme-nvme.8086-50484d42373437363030384b34383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.151767", "end": "2020-11-16 20:19:27.344067", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg 2020-11-16T20:19:28.169 INFO:teuthology.task.ansible.out:_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-sfvFp0-2hQ3-QH8y-3ale-Bo6L-TzSU-RdAfDN", "nvme-INTEL_SSDPED1D480GA_PHMB7476008K480DGN", "nvme-nvme.8086-50484d42373437363030384b34383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:26.192300", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result 2020-11-16T20:19:28.171 INFO:teuthology.task.ansible.out:was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:28.202 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:28 +0000 (0:00:04.833) 0:11:19.617 ******* 2020-11-16T20:19:28.252 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T20:19:28.254 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:28.552 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAc1ROTDuEMXPdJSJUWhk7fn09eKdxgQcZ'], 'uuids': ['acd81826-3709-43bc-a3a7-64f04664ccbd']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.161878", "end": "2020-11-16 20:19:28.525092", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAc1ROTDuEMXPdJSJUWhk7fn09eKdxgQcZ"], "labels": [], "masters": [], "uuids": ["acd81826-3709-43bc-a3a7-64f04664ccbd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:28.363214", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.157223 s, 734 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.157223 s, 734 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:28.565 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:28.567 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:28.664 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtudmd70RDDETifQRk7OzG8yiuum5PoEU3'], 'uuids': ['6ba07774-a6e9-459b-bb1f-9591d067995b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.149373", "end": "2020-11-16 20:19:28.634913", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtudmd70RDDETifQRk7OzG8yiuum5PoEU3"], "labels": [], "masters": [], "uuids": ["6ba07774-a6e9-459b-bb1f-9591d067995b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:27.485540", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk usi 2020-11-16T20:19:28.666 INFO:teuthology.task.ansible.out:ng fdisk or", "other utilities."]} 2020-11-16T20:19:28.676 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:28.697 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:29.013 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T20:19:29.014 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQACn1mF0QzHryzG8l5I80odwwscXhi6cIn'], 'uuids': ['a87edfac-b925-4593-af28-e0146bfa7ceb']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.164639", "end": "2020-11-16 20:19:28.989582", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQACn1mF0QzHryzG8l5I80odwwscXhi6cIn"], "labels": [], "masters": [], "uuids": ["a87edfac-b925-4593-af28-e0146bfa7ceb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:28.824943", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.159902 s, 721 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.159902 s, 721 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:29.025 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5LJ4S', 'wwn-0x5000c500918da5aa'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5LJ4S-part1', 'wwn-0x5000c500918da5aa-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500918da5aa', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5LJ4S', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5LJ4S", "wwn-0x5000c500918da5aa"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5LJ4S-part1", "wwn-0x5000c500918da5aa-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5LJ4S", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918da5aa"}}, "skip_reason": 2020-11-16T20:19:29.027 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2020-11-16T20:19:29.207 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-zd03GV-PIJV-66Au-JSKy-O87e-3lmW-4phd4a', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400QM400BGN', 'nvme-nvme.8086-50484654363230343030514d34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.051304", "end": "2020-11-16 20:19:29.182279", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-zd03GV-PIJV-66Au-JSKy-O87e-3lmW-4phd4a", "nvme-INTEL_SSDPEDMD400G4_PHFT620400QM400BGN", "nvme-nvme.8086-50484654363230343030514d34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:29.130975", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0466254 s, 2.5 GB/s", "std 2020-11-16T20:19:29.209 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0466254 s, 2.5 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:29.220 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:29.231 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:29.487 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAvQHkij04f2RVR3bx5wx87YwINetsuAMS'], 'uuids': ['aa638380-67c6-41a2-aeeb-be24598c3a77']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.143507", "end": "2020-11-16 20:19:29.465879", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAvQHkij04f2RVR3bx5wx87YwINetsuAMS"], "labels": [], "masters": [], "uuids": ["aa638380-67c6-41a2-aeeb-be24598c3a77"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:29.322372", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.141173 s, 817 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.141173 s, 817 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:29.499 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:29.512 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:33.816 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQALQ5OQiHw1Lmsead3YUN98WhBeNoLz3wB'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.160283", "end": "2020-11-16 20:19:33.796055", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQALQ5OQiHw1Lmsead3YUN98WhBeNoLz3wB"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:33.635772", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.155633 s, 741 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.155633 s, 741 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:33.827 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:33.974 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtrJlHV0aMlqLDZvTRPlAbehWKLnS0oJVD'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.110558", "end": "2020-11-16 20:19:29.901336", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtrJlHV0aMlqLDZvTRPlAbehWKLnS0oJVD"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:28.790778", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:33.997 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:34.101 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T20:19:34.103 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAvlbeWMJDJe5uoVT0MCfcJmopYlRVbWSf'], 'uuids': ['ac498e8e-e990-472e-aa27-272f5958531a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.157090", "end": "2020-11-16 20:19:34.076398", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-AQl2Xk2p7j5ct3HK9EBgWrhlch7ssxQAvlbeWMJDJe5uoVT0MCfcJmopYlRVbWSf"], "labels": [], "masters": [], "uuids": ["ac498e8e-e990-472e-aa27-272f5958531a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:33.919308", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152539 s, 756 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152539 s, 756 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:34.143 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:34 +0000 (0:00:05.940) 0:11:25.558 ******* 2020-11-16T20:19:34.409 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-16T20:19:34.411 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.121353", "end": "2020-11-16 20:19:34.380236", "rc": 0, "start": "2020-11-16 20:19:34.258883", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:34.443 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:34 +0000 (0:00:00.299) 0:11:25.858 ******* 2020-11-16T20:19:34.646 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-16T20:19:34.648 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.069879", "end": "2020-11-16 20:19:34.625037", "rc": 0, "start": "2020-11-16 20:19:34.555158", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:34.680 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:34 +0000 (0:00:00.236) 0:11:26.094 ******* 2020-11-16T20:19:34.706 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T20:19:34.742 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:34 +0000 (0:00:00.061) 0:11:26.156 ******* 2020-11-16T20:19:34.769 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T20:19:34.771 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:34.808 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:34 +0000 (0:00:00.066) 0:11:26.222 ******* 2020-11-16T20:19:34.836 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-16T20:19:34.838 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:34.887 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:34 +0000 (0:00:00.078) 0:11:26.301 ******* 2020-11-16T20:19:34.916 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T20:19:34.918 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:34.951 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:34 +0000 (0:00:00.064) 0:11:26.365 ******* 2020-11-16T20:19:34.978 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T20:19:34.979 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:35.011 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.059) 0:11:26.425 ******* 2020-11-16T20:19:35.142 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T20:19:35.144 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T20:19:35.176 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.164) 0:11:26.590 ******* 2020-11-16T20:19:35.213 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T20:19:35.215 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'block_used': 13753462, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341646, 'size_available': 927095382016, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149530, 'block_size': 4096, 'inode_available': 60905446}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341646, "block_size": 4096, "block_total": 240095108, "block_used": 13753462, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905446, "inode_total": 61054976, "inode_used": 149530, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927095382016, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:35.247 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.071) 0:11:26.661 ******* 2020-11-16T20:19:35.274 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-16T20:19:35.276 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:35.307 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.060) 0:11:26.722 ******* 2020-11-16T20:19:35.334 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:35.367 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.059) 0:11:26.781 ******* 2020-11-16T20:19:35.394 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:19:35.396 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:35.428 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.060) 0:11:26.842 ******* 2020-11-16T20:19:35.434 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-16T20:19:35.435 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtuzvZwNV3XVanJdDc2FeMOkrkvaj6Memq'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.296285", "end": "2020-11-16 20:19:35.385309", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtuzvZwNV3XVanJdDc2FeMOkrkvaj6Memq"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:34.089024", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-16T20:19:35.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.055) 0:11:26.897 ******* 2020-11-16T20:19:35.491 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T20:19:35.493 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:35.526 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.042) 0:11:26.940 ******* 2020-11-16T20:19:35.544 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T20:19:35.545 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:35.566 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:35.602 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:35 +0000 (0:00:00.076) 0:11:27.017 ******* 2020-11-16T20:19:43.634 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T20:19:43.636 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHt0arNYU9dY8S9G88CqXe1QcEiu032Ceyv'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.120153", "end": "2020-11-16 20:19:35.755686", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHt0arNYU9dY8S9G88CqXe1QcEiu032Ceyv"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:35.635533", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.115329 s, 1.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.115329 s, 1.0 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:43.637 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": 2020-11-16T20:19:43.638 INFO:teuthology.task.ansible.out:{"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:43.640 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtCJPFJ1r3PlLKH2K4UBvzrZMY6l2HpSTZ'], 'uuids': ['c242352a-d5c3-4f3d-8bb8-ceb7bb191852']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.119150", "end": "2020-11-16 20:19:36.024398", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtCJPFJ1r3PlLKH2K4UBvzrZMY6l2HpSTZ"], "labels": [], "masters": [], "uuids": ["c242352a-d5c3-4f3d-8bb8-ceb7bb191852"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:35.905248", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.114377 s, 1.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.114377 s, 1.0 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:43.640 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2YD', 'wwn-0x5000c500871b846a'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': 2020-11-16T20:19:43.642 INFO:teuthology.task.ansible.out: [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W4J2YD-part1', 'wwn-0x5000c500871b846a-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500871b846a', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W4J2YD', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2YD", "wwn-0x5000c500871b846a"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4J2YD-part1", "wwn-0x5000c500871b846a-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4J2YD", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b846a"}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:43.652 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T20:19:43.653 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:43.684 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:43 +0000 (0:00:08.082) 0:11:35.099 ******* 2020-11-16T20:19:48.352 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T20:19:48.354 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '937703088', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-sfvFp0-2hQ3-QH8y-3ale-Bo6L-TzSU-RdAfDN', 'nvme-INTEL_SSDPED1D480GA_PHMB7476008K480DGN', 'nvme-nvme.8086-50484d42373437363030384b34383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPED1D480GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '447.13 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.055653", "end": "2020-11-16 20:19:48.327678", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-sfvFp0-2hQ3-QH8y-3ale-Bo6L-TzSU-RdAfDN", "nvme-INTEL_SSDPED1D480GA_PHMB7476008K480DGN", "nvme-nvme.8086-50484d42373437363030384b34383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:48.272025", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0508012 s, 2.3 GB/s", "stderr_lines": [" 2020-11-16T20:19:48.355 INFO:teuthology.task.ansible.out:110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0508012 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:48.364 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:48.365 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:48.377 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-16T20:19:48.378 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T20:19:48.410 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:48 +0000 (0:00:04.725) 0:11:39.825 ******* 2020-11-16T20:19:48.603 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T20:19:48.614 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtudmd70RDDETifQRk7OzG8yiuum5PoEU3'], 'uuids': ['6ba07774-a6e9-459b-bb1f-9591d067995b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.111902", "end": "2020-11-16 20:19:48.578456", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtudmd70RDDETifQRk7OzG8yiuum5PoEU3"], "labels": [], "masters": [], "uuids": ["6ba07774-a6e9-459b-bb1f-9591d067995b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:48.466554", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.108483 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.108483 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:48.615 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:48.627 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:48.855 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtrJlHV0aMlqLDZvTRPlAbehWKLnS0oJVD'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.117582", "end": "2020-11-16 20:19:48.833136", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtrJlHV0aMlqLDZvTRPlAbehWKLnS0oJVD"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:48.715554", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.113058 s, 1.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.113058 s, 1.0 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:48.866 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:48.970 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T20:19:48.971 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:49.094 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-16T20:19:49.095 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtuzvZwNV3XVanJdDc2FeMOkrkvaj6Memq'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.113739", "end": "2020-11-16 20:19:49.070339", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-d1WUlSrtjeEq1w0PeGf8jN0Sd7i1kwHtuzvZwNV3XVanJdDc2FeMOkrkvaj6Memq"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-16 20:19:48.956600", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.109057 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.109057 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:49.136 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:49 +0000 (0:00:00.725) 0:11:40.550 ******* 2020-11-16T20:19:49.216 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T20:19:49.218 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:49.448 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:49.459 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2020-11-16T20:19:49.461 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.181173", "end": "2020-11-16 20:19:49.428100", "rc": 0, "start": "2020-11-16 20:19:49.246927", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:49.494 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:49 +0000 (0:00:00.358) 0:11:40.909 ******* 2020-11-16T20:19:49.688 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T20:19:49.690 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:49.700 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-16T20:19:49.701 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.080231", "end": "2020-11-16 20:19:49.682548", "rc": 0, "start": "2020-11-16 20:19:49.602317", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:19:49.733 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:49 +0000 (0:00:00.238) 0:11:41.148 ******* 2020-11-16T20:19:49.760 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-16T20:19:49.794 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:49 +0000 (0:00:00.061) 0:11:41.209 ******* 2020-11-16T20:19:49.821 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2020-11-16T20:19:49.823 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:49.859 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:49 +0000 (0:00:00.064) 0:11:41.274 ******* 2020-11-16T20:19:49.886 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-11-16T20:19:49.888 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:49.921 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:49 +0000 (0:00:00.061) 0:11:41.335 ******* 2020-11-16T20:19:49.938 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T20:19:49.940 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:49.974 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:49 +0000 (0:00:00.053) 0:11:41.389 ******* 2020-11-16T20:19:49.981 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T20:19:49.982 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.019 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.044) 0:11:41.434 ******* 2020-11-16T20:19:50.046 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-11-16T20:19:50.048 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.079 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.059) 0:11:41.494 ******* 2020-11-16T20:19:50.106 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T20:19:50.108 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T20:19:50.144 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.064) 0:11:41.558 ******* 2020-11-16T20:19:50.203 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-16T20:19:50.205 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-16T20:19:50.241 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.096) 0:11:41.655 ******* 2020-11-16T20:19:50.278 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-16T20:19:50.280 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'block_used': 13753475, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226341633, 'size_available': 927095328768, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 149529, 'block_size': 4096, 'inode_available': 60905447}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226341633, "block_size": 4096, "block_total": 240095108, "block_used": 13753475, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60905447, "inode_total": 61054976, "inode_used": 149529, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927095328768, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.314 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.073) 0:11:41.729 ******* 2020-11-16T20:19:50.322 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T20:19:50.324 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T20:19:50.345 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.381 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.066) 0:11:41.795 ******* 2020-11-16T20:19:50.409 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T20:19:50.411 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} TASK [testnode : set_fact] ***************************************************** skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.445 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.064) 0:11:41.860 ******* 2020-11-16T20:19:50.473 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:19:50.474 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.519 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.073) 0:11:41.934 ******* 2020-11-16T20:19:50.528 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T20:19:50.530 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T20:19:50.552 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-16T20:19:50.554 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.590 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.070) 0:11:42.005 ******* 2020-11-16T20:19:50.618 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-16T20:19:50.620 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.655 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.064) 0:11:42.069 ******* 2020-11-16T20:19:50.663 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T20:19:50.663 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T20:19:50.665 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.704 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.049) 0:11:42.119 ******* 2020-11-16T20:19:50.710 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-16T20:19:50.711 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:19:50.758 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.053) 0:11:42.172 ******* 2020-11-16T20:19:50.838 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-16T20:19:50.840 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T20:19:50.874 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.115) 0:11:42.288 ******* 2020-11-16T20:19:50.911 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T20:19:50.913 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T20:19:50.947 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:50 +0000 (0:00:00.072) 0:11:42.361 ******* 2020-11-16T20:19:51.077 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2020-11-16T20:19:51.078 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-16T20:19:51.113 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:51 +0000 (0:00:00.166) 0:11:42.527 ******* 2020-11-16T20:19:55.447 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T20:19:55.448 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true} 2020-11-16T20:19:55.483 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:55 +0000 (0:00:04.369) 0:11:46.897 ******* 2020-11-16T20:19:55.490 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T20:19:55.492 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:55.701 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:55.935 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:55.936 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T20:19:55.968 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:55 +0000 (0:00:00.485) 0:11:47.383 ******* 2020-11-16T20:19:56.113 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T20:19:56.115 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T20:19:56.147 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:56 +0000 (0:00:00.178) 0:11:47.561 ******* 2020-11-16T20:19:56.163 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T20:19:56.165 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:56.279 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T20:19:56.280 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T20:19:56.313 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:56 +0000 (0:00:00.165) 0:11:47.727 ******* 2020-11-16T20:19:56.408 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-16T20:19:56.410 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-16T20:19:56.444 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:56 +0000 (0:00:00.131) 0:11:47.859 ******* 2020-11-16T20:19:56.577 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-11-16T20:19:56.578 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-16T20:19:56.614 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:56 +0000 (0:00:00.169) 0:11:48.028 ******* 2020-11-16T20:19:56.652 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T20:19:56.654 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-16T20:19:56.692 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:19:56 +0000 (0:00:00.078) 0:11:48.106 ******* 2020-11-16T20:19:56.764 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T20:19:56.766 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T20:20:01.124 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-16T20:20:01.157 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:01 +0000 (0:00:04.465) 0:11:52.571 ******* 2020-11-16T20:20:01.196 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-16T20:20:01.198 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-16T20:20:01.210 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:01.250 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:01 +0000 (0:00:00.092) 0:11:52.664 ******* 2020-11-16T20:20:01.383 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-16T20:20:01.385 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T20:20:01.417 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:01 +0000 (0:00:00.167) 0:11:52.832 ******* 2020-11-16T20:20:01.455 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-16T20:20:01.457 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-16T20:20:01.492 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:01 +0000 (0:00:00.074) 0:11:52.906 ******* 2020-11-16T20:20:01.644 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-16T20:20:01.646 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true} 2020-11-16T20:20:01.697 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:01 +0000 (0:00:00.204) 0:11:53.111 ******* 2020-11-16T20:20:01.883 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-16T20:20:01.916 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:01 +0000 (0:00:00.219) 0:11:53.331 ******* 2020-11-16T20:20:02.062 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-16T20:20:02.063 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T20:20:02.098 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:02 +0000 (0:00:00.181) 0:11:53.512 ******* 2020-11-16T20:20:02.105 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T20:20:02.107 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:20:02.142 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:02 +0000 (0:00:00.043) 0:11:53.556 ******* 2020-11-16T20:20:02.233 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-16T20:20:02.235 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-16T20:20:02.270 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:02 +0000 (0:00:00.127) 0:11:53.684 ******* 2020-11-16T20:20:02.309 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-16T20:20:02.311 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.047354", "end": "2020-11-16 20:20:02.290368", "rc": 0, "start": "2020-11-16 20:20:02.243014", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-16T20:20:02.345 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:02 +0000 (0:00:00.074) 0:11:53.759 ******* 2020-11-16T20:20:02.372 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-16T20:20:02.374 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:02.406 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:02 +0000 (0:00:00.061) 0:11:53.821 ******* 2020-11-16T20:20:02.433 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T20:20:02.435 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:02.467 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:02 +0000 (0:00:00.060) 0:11:53.882 ******* 2020-11-16T20:20:02.599 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-16T20:20:02.601 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2020-11-16T20:20:02.634 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:02 +0000 (0:00:00.166) 0:11:54.048 ******* 2020-11-16T20:20:02.968 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-16T20:20:03.003 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:02 +0000 (0:00:00.369) 0:11:54.418 ******* 2020-11-16T20:20:03.316 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:20:03.358 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:03 +0000 (0:00:00.355) 0:11:54.773 ******* 2020-11-16T20:20:03.388 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-16T20:20:03.390 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:03.439 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:03 +0000 (0:00:00.080) 0:11:54.854 ******* 2020-11-16T20:20:03.752 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T20:20:03.754 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-16T20:20:03.800 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:03 +0000 (0:00:00.360) 0:11:55.214 ******* 2020-11-16T20:20:03.827 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T20:20:03.829 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:03.866 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:03 +0000 (0:00:00.066) 0:11:55.281 ******* 2020-11-16T20:20:03.926 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2020-11-16T20:20:03.927 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:20:03.962 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:03 +0000 (0:00:00.095) 0:11:55.376 ******* 2020-11-16T20:20:04.065 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T20:20:04.068 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 20:07:36 UTC", "ActiveEnterTimestampMonotonic": "28004520", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount systemd-tmpfiles-setup.service basic.target systemd-journald.socket system.slice sysinit.target network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 20:07:30 UTC", "AssertTimestampMonotonic": "21316056", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 20:07:30 UTC", "ConditionTimestampMonotonic": "21316056", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-16T20:20:04.069 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1626", "ExecMainStartTimestamp": "Mon 2020-11-16 20:07:36 UTC", "ExecMainStartTimestampMonotonic": "28004507", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 20:07:30 UTC", "InactiveExitTimestampMonotonic": "21317408", "InvocationID": "306cbaf77c5648119eeb98075483a5ab", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-11-16T20:20:04.071 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1626", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 20:07:36 UTC", "S 2020-11-16T20:20:04.072 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "28004520", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 20:07:36 UTC", "WatchdogTimestampMonotonic": "28004518", "WatchdogUSec": "0"}} 2020-11-16T20:20:04.103 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:04 +0000 (0:00:00.140) 0:11:55.517 ******* 2020-11-16T20:20:04.130 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:20:04.132 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T20:20:04.164 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:04 +0000 (0:00:00.061) 0:11:55.578 ******* 2020-11-16T20:20:04.171 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-16T20:20:04.173 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.049180", "end": "2020-11-16 20:20:04.128822", "rc": 0, "start": "2020-11-16 20:20:04.079642", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-16T20:20:04.209 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:04 +0000 (0:00:00.045) 0:11:55.624 ******* 2020-11-16T20:20:04.216 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:20:04.217 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:04.251 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:04 +0000 (0:00:00.042) 0:11:55.666 ******* 2020-11-16T20:20:04.259 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-11-16T20:20:04.260 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:04.295 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:04 +0000 (0:00:00.043) 0:11:55.710 ******* 2020-11-16T20:20:04.304 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T20:20:04.306 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:04.344 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:04 +0000 (0:00:00.048) 0:11:55.758 ******* 2020-11-16T20:20:04.354 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-16T20:20:04.356 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:04.395 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:04 +0000 (0:00:00.051) 0:11:55.809 ******* 2020-11-16T20:20:05.227 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T20:20:05.229 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1605557054, "cache_updated": false, "changed": false} 2020-11-16T20:20:05.260 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:05 +0000 (0:00:00.865) 0:11:56.675 ******* 2020-11-16T20:20:05.267 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-16T20:20:05.268 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:20:05.301 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:05 +0000 (0:00:00.040) 0:11:56.715 ******* 2020-11-16T20:20:05.309 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T20:20:05.310 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:05.342 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:05 +0000 (0:00:00.040) 0:11:56.756 ******* 2020-11-16T20:20:05.349 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-16T20:20:05.350 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:05.386 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:05 +0000 (0:00:00.043) 0:11:56.800 ******* 2020-11-16T20:20:05.466 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-11-16T20:20:05.468 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T20:20:05.501 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:05 +0000 (0:00:00.115) 0:11:56.916 ******* 2020-11-16T20:20:05.700 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-16T20:20:05.701 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-16T20:20:05.735 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:05 +0000 (0:00:00.233) 0:11:57.150 ******* 2020-11-16T20:20:09.844 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T20:20:09.846 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-16T20:20:09.878 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:09 +0000 (0:00:04.142) 0:12:01.293 ******* 2020-11-16T20:20:09.885 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-16T20:20:09.887 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:09.920 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:09 +0000 (0:00:00.041) 0:12:01.335 ******* 2020-11-16T20:20:14.402 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-16T20:20:14.403 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2020-11-16 20:07:24 UTC", "ActiveEnterTimestampMonotonic": "24441013", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target systemd-journald.socket systemd-tmpfiles-setup.service system.slice network.target -.mount sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Mon 2020-11-16 20:07:17 UTC", "AssertTimestampMonotonic": "17844174", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2020-11-16 20:07:17 UTC", "ConditionTimestampMonotonic": "17844173", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2020-11-16T20:20:14.404 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1729", "ExecMainStartTimestamp": "Mon 2020-11-16 20:07:24 UTC", "ExecMainStartTimestampMonotonic": "24440999", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2020-11-16 20:07:17 UTC", "InactiveExitTimestampMonotonic": "17845054", "InvocationID": "21ad073c78e94d7fb481113e098c8e8a", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128053", "LimitNPROCSoft": "128053", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2020-11-16T20:20:14.406 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128053", "LimitSIGPENDINGSoft": "128053", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1729", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Mon 2020-11-16 20:07:24 UTC", "S 2020-11-16T20:20:14.407 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "24441013", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Mon 2020-11-16 20:07:24 UTC", "WatchdogTimestampMonotonic": "24441011", "WatchdogUSec": "0"}} 2020-11-16T20:20:14.438 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:14 +0000 (0:00:04.517) 0:12:05.852 ******* 2020-11-16T20:20:14.465 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:20:14.467 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-16T20:20:14.498 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:14 +0000 (0:00:00.060) 0:12:05.913 ******* 2020-11-16T20:20:14.525 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:20:14.527 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:14.559 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:14 +0000 (0:00:00.060) 0:12:05.973 ******* 2020-11-16T20:20:14.596 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-16T20:20:14.598 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:14.632 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:14 +0000 (0:00:00.073) 0:12:06.047 ******* 2020-11-16T20:20:15.186 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-16T20:20:15.187 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T20:20:15.221 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:15 +0000 (0:00:00.588) 0:12:06.635 ******* 2020-11-16T20:20:15.552 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-16T20:20:15.553 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"cache_update_time": 1605557098, "cache_updated": false, "changed": false} 2020-11-16T20:20:15.586 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:15 +0000 (0:00:00.365) 0:12:07.000 ******* 2020-11-16T20:20:15.615 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-16T20:20:15.617 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:15.649 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:15 +0000 (0:00:00.063) 0:12:07.064 ******* 2020-11-16T20:20:15.709 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [smithi092.front.sepia.ceph.com] => {"changed": false} 2020-11-16T20:20:15.744 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:15 +0000 (0:00:00.094) 0:12:07.158 ******* 2020-11-16T20:20:15.771 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T20:20:15.773 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-16T20:20:15.805 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:15 +0000 (0:00:00.061) 0:12:07.220 ******* 2020-11-16T20:20:15.838 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:15 +0000 (0:00:00.031) 0:12:07.252 ******* 2020-11-16T20:20:20.161 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T20:20:20.162 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605557031.3474064, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605557031.3434064, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605557031.3434064, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-16T20:20:20.194 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:20 +0000 (0:00:04.356) 0:12:11.609 ******* 2020-11-16T20:20:20.336 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-11-16T20:20:20.338 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-16T20:20:20.370 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:20 +0000 (0:00:00.175) 0:12:11.784 ******* 2020-11-16T20:20:20.576 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T20:20:20.577 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-16T20:20:20.616 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:20 +0000 (0:00:00.246) 0:12:12.030 ******* 2020-11-16T20:20:20.643 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T20:20:20.645 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T20:20:20.676 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:20 +0000 (0:00:00.060) 0:12:12.091 ******* 2020-11-16T20:20:20.706 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T20:20:20.707 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:20.742 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:20 +0000 (0:00:00.065) 0:12:12.156 ******* 2020-11-16T20:20:20.877 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T20:20:20.878 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T20:20:20.910 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:20 +0000 (0:00:00.168) 0:12:12.325 ******* 2020-11-16T20:20:21.032 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T20:20:21.034 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T20:20:21.072 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:21 +0000 (0:00:00.161) 0:12:12.487 ******* 2020-11-16T20:20:21.100 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T20:20:21.102 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.135 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:21 +0000 (0:00:00.063) 0:12:12.550 ******* 2020-11-16T20:20:21.175 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T20:20:21.176 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} TASK [testnode : Install packages via pip] ************************************* 2020-11-16T20:20:21.177 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-16T20:20:21.177 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi092.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.211 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:21 +0000 (0:00:00.075) 0:12:12.625 ******* 2020-11-16T20:20:21.218 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.220 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.254 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:21 +0000 (0:00:00.043) 0:12:12.668 ******* 2020-11-16T20:20:21.291 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T20:20:21.293 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.303 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.305 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.315 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.349 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:21 +0000 (0:00:00.095) 0:12:12.764 ******* 2020-11-16T20:20:21.376 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:20:21.378 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.414 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:21 +0000 (0:00:00.064) 0:12:12.828 ******* 2020-11-16T20:20:21.442 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T20:20:21.476 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.478 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:21 +0000 (0:00:00.063) 0:12:12.892 ******* 2020-11-16T20:20:21.505 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T20:20:21.507 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:21.558 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:21 +0000 (0:00:00.080) 0:12:12.972 ******* 2020-11-16T20:20:25.774 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-16T20:20:25.776 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false} 2020-11-16T20:20:25.923 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:25 +0000 (0:00:04.365) 0:12:17.337 ******* 2020-11-16T20:20:25.931 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T20:20:25.933 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:25.967 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:25 +0000 (0:00:00.043) 0:12:17.381 ******* 2020-11-16T20:20:25.973 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2020-11-16T20:20:25.975 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2020-11-16T20:20:26.010 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.043) 0:12:17.425 ******* 2020-11-16T20:20:26.018 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:26.056 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.045) 0:12:17.470 ******* 2020-11-16T20:20:26.083 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T20:20:26.085 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:26.120 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.064) 0:12:17.534 ******* 2020-11-16T20:20:26.147 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:26.184 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.063) 0:12:17.598 ******* 2020-11-16T20:20:26.191 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-16T20:20:26.193 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605556984.9419005, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605556984.9379027, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605556984.9379027, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 67, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-16T20:20:26.226 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.042) 0:12:17.641 ******* 2020-11-16T20:20:26.234 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T20:20:26.236 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:26.270 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.043) 0:12:17.685 ******* 2020-11-16T20:20:26.375 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-16T20:20:26.376 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-16T20:20:26.410 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.139) 0:12:17.825 ******* 2020-11-16T20:20:26.429 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T20:20:26.431 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.009655", "end": "2020-11-16 20:20:26.394442", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 20:20:26.384787", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:20:26.441 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-11-16T20:20:26.442 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-16T20:20:26.475 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.063) 0:12:17.889 ******* 2020-11-16T20:20:26.503 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-16T20:20:26.505 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:26.537 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:26 +0000 (0:00:00.062) 0:12:17.952 ******* 2020-11-16T20:20:26.545 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T20:20:26.547 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.005856", "end": "2020-11-16 20:20:26.514917", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 20:20:26.509061", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:20:26.663 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005511", "end": "2020-11-16 20:20:26.634410", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 20:20:26.628899", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T20:20:26.674 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-16T20:20:26.676 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-16T20:20:30.876 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:30 +0000 (0:00:04.337) 0:12:22.290 ******* 2020-11-16T20:20:30.992 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T20:20:30.993 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.032741", "end": "2020-11-16 20:20:26.776825", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 20:20:26.744084", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:20:31.033 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:31 +0000 (0:00:00.157) 0:12:22.448 ******* 2020-11-16T20:20:31.041 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2020-11-16T20:20:31.043 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-16T20:20:31.077 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:31 +0000 (0:00:00.044) 0:12:22.492 ******* 2020-11-16T20:20:31.085 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T20:20:31.121 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:31 +0000 (0:00:00.043) 0:12:22.536 ******* 2020-11-16T20:20:31.128 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-16T20:20:31.130 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.175 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:31 +0000 (0:00:00.053) 0:12:22.590 ******* 2020-11-16T20:20:31.224 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-16T20:20:31.226 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.237 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.238 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.274 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:31 +0000 (0:00:00.098) 0:12:22.689 ******* 2020-11-16T20:20:31.281 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T20:20:31.284 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-16T20:20:31.320 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-16T20:20:31.322 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.356 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.358 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi013.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.390 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:31 +0000 (0:00:00.115) 0:12:22.805 ******* 2020-11-16T20:20:31.418 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-16T20:20:31.419 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.455 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:31 +0000 (0:00:00.064) 0:12:22.869 ******* 2020-11-16T20:20:31.483 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-16T20:20:31.485 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:31.517 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:31 +0000 (0:00:00.061) 0:12:22.931 ******* 2020-11-16T20:20:35.558 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-11-16T20:20:35.560 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:35.593 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:35 +0000 (0:00:04.076) 0:12:27.008 ******* 2020-11-16T20:20:35.620 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-16T20:20:35.622 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:35.657 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:35 +0000 (0:00:00.063) 0:12:27.071 ******* 2020-11-16T20:20:35.684 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-16T20:20:35.686 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:35.720 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:35 +0000 (0:00:00.063) 0:12:27.135 ******* 2020-11-16T20:20:35.747 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T20:20:35.749 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:35.784 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:35 +0000 (0:00:00.063) 0:12:27.198 ******* 2020-11-16T20:20:35.811 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-16T20:20:35.813 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:35.849 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:35 +0000 (0:00:00.063) 0:12:27.262 ******* 2020-11-16T20:20:35.875 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-16T20:20:35.877 INFO:teuthology.task.ansible.out:skipping: [smithi013.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-16T20:20:35.909 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:35 +0000 (0:00:00.061) 0:12:27.324 ******* 2020-11-16T20:20:36.073 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-16T20:20:36.074 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011063", "end": "2020-11-16 20:20:36.039736", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-16 20:20:36.028673", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:20:36.190 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006122", "end": "2020-11-16 20:20:36.166387", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-16 20:20:36.160265", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:20:36.319 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005673", "end": "2020-11-16 20:20:36.290508", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-16 20:20:36.284835", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-16T20:20:36.455 INFO:teuthology.task.ansible.out:ok: [smithi013.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.035002", "end": "2020-11-16 20:20:36.435598", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-16 20:20:36.400596", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-16T20:20:36.496 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:36 +0000 (0:00:00.587) 0:12:27.911 ******* 2020-11-16T20:20:36.498 INFO:teuthology.task.ansible.out: 2020-11-16T20:20:36.525 INFO:teuthology.task.ansible.out: 2020-11-16T20:20:36.527 INFO:teuthology.task.ansible.out:TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-16T20:20:36.561 INFO:teuthology.task.ansible.out:Monday 16 November 2020 20:20:36 +0000 (0:00:00.064) 0:12:27.975 ******* 2020-11-16T20:20:36.692 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-11-16T20:20:36.694 INFO:teuthology.task.ansible.out:changed: [smithi013.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-16T20:20:36.788 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-11-16T20:20:36.795 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-11-16T20:20:36.799 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi013.front.sepia.ceph.com : ok=108 changed=27 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 2020-11-16T20:20:36.801 INFO:teuthology.task.ansible.out:smithi092.front.sepia.ceph.com : ok=108 changed=27 unreachable=0 failed=0 skipped=181 rescued=0 ignored=0 Monday 16 November 2020 20:20:36 +0000 (0:00:00.238) 0:12:28.214 ******* =============================================================================== common : Set have_entitlements ---------------------------------------- 161.60s users : Check keys_repo HEAD sha1 -------------------------------------- 38.40s users : Create all admin users with sudo access. ----------------------- 16.59s Gathering Facts -------------------------------------------------------- 13.52s ansible-managed : Create the sudo group. ------------------------------- 12.66s testnode : Make sure apt dependencies are installed --------------------- 9.10s testnode : Set the system's hostname ------------------------------------ 8.74s testnode : Check for teuthology-lock command ---------------------------- 8.43s common : List CDN releases available to system -------------------------- 8.37s testnode : Use perl to add user_xattr to the root mount options in fstab. --- 8.29s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 8.16s testnode : Upgrade packages --------------------------------------------- 8.11s common : Update apt cache ----------------------------------------------- 5.88s Gathering Facts --------------------------------------------------------- 5.81s testnode : Ensure packages are not present. ----------------------------- 5.52s testnode : Compile list of non-root partitions -------------------------- 4.87s testnode : Zap all non-root disks --------------------------------------- 4.83s ansible-managed : Delete the ansible users password. -------------------- 4.58s common : Get the current timezone --------------------------------------- 4.53s testnode : Make sure ntpd is running. ----------------------------------- 4.52s 2020-11-16T20:20:37.171 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi013.front.sepia.ceph.com')] 2020-11-16T20:20:37.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T20:20:37.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:20:37.250 INFO:teuthology.orchestra.run.smithi092:> true 2020-11-16T20:20:37.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-16T20:20:37.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi013.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-16T20:20:37.523 INFO:teuthology.orchestra.run.smithi013:> true 2020-11-16T20:20:37.715 INFO:teuthology.run_tasks:Running task clock... 2020-11-16T20:20:37.725 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-16T20:20:37.726 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T20:20:37.726 INFO:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T20:20:37.786 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-16T20:20:37.786 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: Command line: ntpd -gq 2020-11-16T20:20:37.787 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: proto: precision = 0.123 usec (-23) 2020-11-16T20:20:37.788 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T20:20:37.788 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: restrict ::: KOD does nothing without LIMITED. 2020-11-16T20:20:37.788 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: Listen and drop on 0 v6wildcard [::]:123 2020-11-16T20:20:37.788 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-16T20:20:37.788 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: Listen normally on 2 lo 127.0.0.1:123 2020-11-16T20:20:37.788 INFO:teuthology.orchestra.run.smithi092.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T20:20:37.789 INFO:teuthology.orchestra.run.smithi092.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-16T20:20:37.789 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: Listen normally on 3 enp3s0f1 172.21.15.92:123 2020-11-16T20:20:37.789 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: Listen normally on 4 lo [::1]:123 2020-11-16T20:20:37.790 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cd45%5]:123 2020-11-16T20:20:37.790 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:37 ntpd[5653]: Listening on routing socket on fd #22 for interface updates 2020-11-16T20:20:38.786 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:38 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:38.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:38 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:38.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:38 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:38.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:38 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:39 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:39 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:39 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:39 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:40.786 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:40 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:40.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:40 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:40.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:40 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:40.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:40 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:41.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:41.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:41.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:41.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:41.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:41.789 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:41.789 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:41.789 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:41.789 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:41.790 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:41.790 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:41.790 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:41 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:42 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:42 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:42 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:42 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:43.786 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:43.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:43.788 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:43 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:44.786 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:44 ntpd[5653]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:44.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:44 ntpd[5653]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:44.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:44 ntpd[5653]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:44.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:44 ntpd[5653]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:44.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:44 ntpd[5653]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-16T20:20:44.787 INFO:teuthology.orchestra.run.smithi092.stderr:16 Nov 20:20:44 ntpd[5653]: can't open /var/log/ntpstats/loopstats.20201116: Permission denied 2020-11-16T20:20:44.787 INFO:teuthology.orchestra.run.smithi092.stdout:16 Nov 20:20:44 ntpd[5653]: ntpd: time slew +0.001085 s 2020-11-16T20:20:44.788 INFO:teuthology.orchestra.run.smithi092.stdout:ntpd: time slew +0.001085s 2020-11-16T20:20:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: remote refid st t when poll reach delay offset jitter 2020-11-16T20:20:44.967 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================== 2020-11-16T20:20:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T20:20:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T20:20:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T20:20:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T20:20:44.968 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-16T20:20:44.969 INFO:teuthology.orchestra.run.smithi013:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-16T20:20:45.032 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-16T20:20:45.032 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: Command line: ntpd -gq 2020-11-16T20:20:45.033 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: proto: precision = 0.126 usec (-23) 2020-11-16T20:20:45.033 INFO:teuthology.orchestra.run.smithi013.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T20:20:45.034 INFO:teuthology.orchestra.run.smithi013.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-16T20:20:45.034 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-16T20:20:45.034 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: restrict ::: KOD does nothing without LIMITED. 2020-11-16T20:20:45.034 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: Listen and drop on 0 v6wildcard [::]:123 2020-11-16T20:20:45.034 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-16T20:20:45.035 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: Listen normally on 2 lo 127.0.0.1:123 2020-11-16T20:20:45.035 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: Listen normally on 3 ens1f0 172.21.15.13:123 2020-11-16T20:20:45.035 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: Listen normally on 4 lo [::1]:123 2020-11-16T20:20:45.035 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:12c6%3]:123 2020-11-16T20:20:45.036 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:45 ntpd[5750]: Listening on routing socket on fd #22 for interface updates 2020-11-16T20:20:46.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:46 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:46.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:46 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:46.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:46 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:46.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:46 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:46.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:46 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:46.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:46 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:46.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:46 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:46.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:46 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:48.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:48.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:48.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:48.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:48.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:48.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:48.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:48.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:48.035 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:48.035 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:48.035 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:48.035 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:48 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:49.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:49 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:49.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:49 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:49.035 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:49 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:49.035 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:49 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:50.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:50.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:50.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:50.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:50.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:50.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:50.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:50.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:50.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:50.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:50.035 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:50.035 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:50 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:51.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:51 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:51.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:51 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:51.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:51 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:51.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:51 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:52.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:52 ntpd[5750]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-16T20:20:52.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:52 ntpd[5750]: can't open /var/log/ntpstats/rawstats.20201116: Permission denied 2020-11-16T20:20:52.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:52 ntpd[5750]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-16T20:20:52.033 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:52 ntpd[5750]: can't open /var/log/ntpstats/peerstats.20201116: Permission denied 2020-11-16T20:20:52.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:52 ntpd[5750]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-16T20:20:52.034 INFO:teuthology.orchestra.run.smithi013.stderr:16 Nov 20:20:52 ntpd[5750]: can't open /var/log/ntpstats/loopstats.20201116: Permission denied 2020-11-16T20:20:52.034 INFO:teuthology.orchestra.run.smithi013.stdout:16 Nov 20:20:52 ntpd[5750]: ntpd: time slew -0.005353 s 2020-11-16T20:20:52.034 INFO:teuthology.orchestra.run.smithi013.stdout:ntpd: time slew -0.005353s 2020-11-16T20:20:52.082 INFO:teuthology.orchestra.run.smithi013.stdout: remote refid st t when poll reach delay offset jitter 2020-11-16T20:20:52.082 INFO:teuthology.orchestra.run.smithi013.stdout:============================================================================== 2020-11-16T20:20:52.083 INFO:teuthology.orchestra.run.smithi013.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T20:20:52.083 INFO:teuthology.orchestra.run.smithi013.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T20:20:52.083 INFO:teuthology.orchestra.run.smithi013.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T20:20:52.083 INFO:teuthology.orchestra.run.smithi013.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-16T20:20:52.083 INFO:teuthology.run_tasks:Running task install... 2020-11-16T20:20:52.092 DEBUG:teuthology.task.install:project ceph 2020-11-16T20:20:52.093 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '911cc35794d871071a8bebfb0919482263b1d18b'}} 2020-11-16T20:20:52.093 DEBUG:teuthology.task.install:config {'sha1': '911cc35794d871071a8bebfb0919482263b1d18b'} 2020-11-16T20:20:52.093 INFO:teuthology.task.install:Using flavor: basic 2020-11-16T20:20:52.099 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-16T20:20:52.100 INFO:teuthology.task.install:extra packages: [] 2020-11-16T20:20:52.100 INFO:teuthology.orchestra.run.smithi092:> sudo lsb_release -is 2020-11-16T20:20:52.148 INFO:teuthology.orchestra.run.smithi092.stdout:Ubuntu 2020-11-16T20:20:52.153 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T20:20:52.154 INFO:teuthology.orchestra.run.smithi013:> sudo lsb_release -is 2020-11-16T20:20:52.154 INFO:teuthology.orchestra.run.smithi092:> sudo apt-key list | grep Ceph 2020-11-16T20:20:52.208 INFO:teuthology.orchestra.run.smithi013.stdout:Ubuntu 2020-11-16T20:20:52.246 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T20:20:52.246 INFO:teuthology.orchestra.run.smithi013:> sudo apt-key list | grep Ceph 2020-11-16T20:20:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-16T20:20:52.303 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-16T20:20:52.303 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph.com (release key) 2020-11-16T20:20:52.304 INFO:teuthology.orchestra.run.smithi092:> sudo lsb_release -is 2020-11-16T20:20:52.354 INFO:teuthology.orchestra.run.smithi092.stdout:Ubuntu 2020-11-16T20:20:52.358 INFO:teuthology.orchestra.run.smithi013.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-16T20:20:52.359 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T20:20:52.359 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2020-11-16T20:20:52.360 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=911cc35794d871071a8bebfb0919482263b1d18b 2020-11-16T20:20:52.413 INFO:teuthology.orchestra.run.smithi013.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-16T20:20:52.414 INFO:teuthology.orchestra.run.smithi013.stdout:uid [ unknown] Ceph.com (release key) 2020-11-16T20:20:52.415 INFO:teuthology.orchestra.run.smithi013:> sudo lsb_release -is 2020-11-16T20:20:52.467 INFO:teuthology.orchestra.run.smithi013.stdout:Ubuntu 2020-11-16T20:20:52.473 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T20:20:52.473 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2020-11-16T20:20:52.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=911cc35794d871071a8bebfb0919482263b1d18b 2020-11-16T20:20:52.533 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default/ 2020-11-16T20:20:52.533 INFO:teuthology.task.install.deb:Package version is 16.0.0-7219-g911cc357-1bionic 2020-11-16T20:20:52.645 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default/ 2020-11-16T20:20:52.646 INFO:teuthology.task.install.deb:Package version is 16.0.0-7219-g911cc357-1bionic 2020-11-16T20:20:52.665 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:20:52.666 INFO:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T20:20:52.671 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T20:20:52.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:20:52.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:20:52.680 INFO:teuthology.orchestra.run.smithi092.stderr:168 bytes copied, 6.6738e-05 s, 2.5 MB/s 2020-11-16T20:20:52.681 INFO:teuthology.orchestra.run.smithi092:> sudo apt-get update 2020-11-16T20:20:52.772 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:20:52.772 INFO:teuthology.orchestra.run.smithi013:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T20:20:52.777 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-16T20:20:52.786 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:20:52.786 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:20:52.787 INFO:teuthology.orchestra.run.smithi013.stderr:168 bytes copied, 7.3263e-05 s, 2.3 MB/s 2020-11-16T20:20:52.787 INFO:teuthology.orchestra.run.smithi013:> sudo apt-get update 2020-11-16T20:20:52.969 INFO:teuthology.orchestra.run.smithi013.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-16T20:20:52.989 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T20:20:52.989 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-11-16T20:20:52.990 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic InRelease 2020-11-16T20:20:53.013 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-16T20:20:53.037 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-16T20:20:53.056 INFO:teuthology.orchestra.run.smithi013.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic InRelease 2020-11-16T20:20:53.057 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-16T20:20:53.060 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,954 B] 2020-11-16T20:20:53.079 INFO:teuthology.orchestra.run.smithi013.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-16T20:20:53.080 INFO:teuthology.orchestra.run.smithi013.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-16T20:20:53.095 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-16T20:20:53.105 INFO:teuthology.orchestra.run.smithi013.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-16T20:20:53.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,954 B] 2020-11-16T20:20:53.159 INFO:teuthology.orchestra.run.smithi013.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-16T20:20:53.167 INFO:teuthology.orchestra.run.smithi013.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-16T20:20:53.315 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-16T20:20:53.419 INFO:teuthology.orchestra.run.smithi013.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-16T20:20:53.643 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 279 kB in 1s (368 kB/s) 2020-11-16T20:20:53.692 INFO:teuthology.orchestra.run.smithi013.stdout:Fetched 190 kB in 1s (272 kB/s) 2020-11-16T20:20:54.882 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2020-11-16T20:20:54.929 INFO:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7219-g911cc357-1bionic cephadm=16.0.0-7219-g911cc357-1bionic ceph-mds=16.0.0-7219-g911cc357-1bionic ceph-mgr=16.0.0-7219-g911cc357-1bionic ceph-common=16.0.0-7219-g911cc357-1bionic ceph-fuse=16.0.0-7219-g911cc357-1bionic ceph-test=16.0.0-7219-g911cc357-1bionic radosgw=16.0.0-7219-g911cc357-1bionic python3-rados=16.0.0-7219-g911cc357-1bionic python3-rgw=16.0.0-7219-g911cc357-1bionic python3-cephfs=16.0.0-7219-g911cc357-1bionic python3-rbd=16.0.0-7219-g911cc357-1bionic libcephfs2=16.0.0-7219-g911cc357-1bionic libcephfs-dev=16.0.0-7219-g911cc357-1bionic librados2=16.0.0-7219-g911cc357-1bionic librbd1=16.0.0-7219-g911cc357-1bionic rbd-fuse=16.0.0-7219-g911cc357-1bionic 2020-11-16T20:20:55.010 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2020-11-16T20:20:55.017 INFO:teuthology.orchestra.run.smithi013.stdout:Reading package lists... 2020-11-16T20:20:55.068 INFO:teuthology.orchestra.run.smithi013:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7219-g911cc357-1bionic cephadm=16.0.0-7219-g911cc357-1bionic ceph-mds=16.0.0-7219-g911cc357-1bionic ceph-mgr=16.0.0-7219-g911cc357-1bionic ceph-common=16.0.0-7219-g911cc357-1bionic ceph-fuse=16.0.0-7219-g911cc357-1bionic ceph-test=16.0.0-7219-g911cc357-1bionic radosgw=16.0.0-7219-g911cc357-1bionic python3-rados=16.0.0-7219-g911cc357-1bionic python3-rgw=16.0.0-7219-g911cc357-1bionic python3-cephfs=16.0.0-7219-g911cc357-1bionic python3-rbd=16.0.0-7219-g911cc357-1bionic libcephfs2=16.0.0-7219-g911cc357-1bionic libcephfs-dev=16.0.0-7219-g911cc357-1bionic librados2=16.0.0-7219-g911cc357-1bionic librbd1=16.0.0-7219-g911cc357-1bionic rbd-fuse=16.0.0-7219-g911cc357-1bionic 2020-11-16T20:20:55.162 INFO:teuthology.orchestra.run.smithi013.stdout:Reading package lists... 2020-11-16T20:20:55.240 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2020-11-16T20:20:55.241 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2020-11-16T20:20:55.405 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2020-11-16T20:20:55.405 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-16T20:20:55.406 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-16T20:20:55.406 INFO:teuthology.orchestra.run.smithi092.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-16T20:20:55.406 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-16T20:20:55.406 INFO:teuthology.orchestra.run.smithi092.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-16T20:20:55.406 INFO:teuthology.orchestra.run.smithi092.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T20:20:55.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-16T20:20:55.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2020-11-16T20:20:55.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-11-16T20:20:55.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-11-16T20:20:55.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-16T20:20:55.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-16T20:20:55.407 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-16T20:20:55.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-16T20:20:55.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-16T20:20:55.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2020-11-16T20:20:55.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-16T20:20:55.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T20:20:55.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T20:20:55.408 INFO:teuthology.orchestra.run.smithi092.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T20:20:55.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T20:20:55.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-16T20:20:55.409 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2020-11-16T20:20:55.409 INFO:teuthology.orchestra.run.smithi092.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-16T20:20:55.409 INFO:teuthology.orchestra.run.smithi092.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-16T20:20:55.410 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-16T20:20:55.410 INFO:teuthology.orchestra.run.smithi092.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-16T20:20:55.410 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-16T20:20:55.410 INFO:teuthology.orchestra.run.smithi092.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-16T20:20:55.410 INFO:teuthology.orchestra.run.smithi092.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-16T20:20:55.410 INFO:teuthology.orchestra.run.smithi092.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-16T20:20:55.411 INFO:teuthology.orchestra.run.smithi092.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-16T20:20:55.411 INFO:teuthology.orchestra.run.smithi092.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-16T20:20:55.411 INFO:teuthology.orchestra.run.smithi092.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-16T20:20:55.411 INFO:teuthology.orchestra.run.smithi092.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-16T20:20:55.411 INFO:teuthology.orchestra.run.smithi092.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-16T20:20:55.411 INFO:teuthology.orchestra.run.smithi092.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-16T20:20:55.411 INFO:teuthology.orchestra.run.smithi092.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-16T20:20:55.485 INFO:teuthology.orchestra.run.smithi013.stdout:Building dependency tree... 2020-11-16T20:20:55.485 INFO:teuthology.orchestra.run.smithi013.stdout:Reading state information... 2020-11-16T20:20:55.507 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2020-11-16T20:20:55.507 INFO:teuthology.orchestra.run.smithi092.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-16T20:20:55.507 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-16T20:20:55.508 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2020-11-16T20:20:55.508 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-11-16T20:20:55.509 INFO:teuthology.orchestra.run.smithi092.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-11-16T20:20:55.509 INFO:teuthology.orchestra.run.smithi092.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-11-16T20:20:55.509 INFO:teuthology.orchestra.run.smithi092.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-11-16T20:20:55.509 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-11-16T20:20:55.509 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cryptography 2020-11-16T20:20:55.510 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-16T20:20:55.510 INFO:teuthology.orchestra.run.smithi092.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-16T20:20:55.510 INFO:teuthology.orchestra.run.smithi092.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-16T20:20:55.514 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-16T20:20:55.514 INFO:teuthology.orchestra.run.smithi092.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-16T20:20:55.514 INFO:teuthology.orchestra.run.smithi092.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-16T20:20:55.514 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-16T20:20:55.515 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-16T20:20:55.515 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyinotify python3-pyparsing python3-pytest python3-rados python3-rbd 2020-11-16T20:20:55.515 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze.lru python3-rgw python3-routes python3-rsa python3-scipy 2020-11-16T20:20:55.515 INFO:teuthology.orchestra.run.smithi092.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T20:20:55.515 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T20:20:55.515 INFO:teuthology.orchestra.run.smithi092.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T20:20:55.515 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T20:20:55.515 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-16T20:20:55.516 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages will be upgraded: 2020-11-16T20:20:55.516 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 librbd1 2020-11-16T20:20:55.680 INFO:teuthology.orchestra.run.smithi013.stdout:The following additional packages will be installed: 2020-11-16T20:20:55.680 INFO:teuthology.orchestra.run.smithi013.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-16T20:20:55.681 INFO:teuthology.orchestra.run.smithi013.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-16T20:20:55.681 INFO:teuthology.orchestra.run.smithi013.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-16T20:20:55.681 INFO:teuthology.orchestra.run.smithi013.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-16T20:20:55.681 INFO:teuthology.orchestra.run.smithi013.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-16T20:20:55.682 INFO:teuthology.orchestra.run.smithi013.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-16T20:20:55.682 INFO:teuthology.orchestra.run.smithi013.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-16T20:20:55.682 INFO:teuthology.orchestra.run.smithi013.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2020-11-16T20:20:55.682 INFO:teuthology.orchestra.run.smithi013.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2020-11-16T20:20:55.682 INFO:teuthology.orchestra.run.smithi013.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2020-11-16T20:20:55.682 INFO:teuthology.orchestra.run.smithi013.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-16T20:20:55.683 INFO:teuthology.orchestra.run.smithi013.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-16T20:20:55.683 INFO:teuthology.orchestra.run.smithi013.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-16T20:20:55.683 INFO:teuthology.orchestra.run.smithi013.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-16T20:20:55.683 INFO:teuthology.orchestra.run.smithi013.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-16T20:20:55.683 INFO:teuthology.orchestra.run.smithi013.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2020-11-16T20:20:55.683 INFO:teuthology.orchestra.run.smithi013.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-16T20:20:55.683 INFO:teuthology.orchestra.run.smithi013.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T20:20:55.683 INFO:teuthology.orchestra.run.smithi013.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T20:20:55.684 INFO:teuthology.orchestra.run.smithi013.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T20:20:55.684 INFO:teuthology.orchestra.run.smithi013.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T20:20:55.684 INFO:teuthology.orchestra.run.smithi013.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-16T20:20:55.685 INFO:teuthology.orchestra.run.smithi092.stdout:2 upgraded, 110 newly installed, 0 to remove and 1 not upgraded. 2020-11-16T20:20:55.685 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 168 MB of archives. 2020-11-16T20:20:55.685 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-16T20:20:55.685 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7219-g911cc357-1bionic [1,921 kB] 2020-11-16T20:20:55.685 INFO:teuthology.orchestra.run.smithi013.stdout:Suggested packages: 2020-11-16T20:20:55.685 INFO:teuthology.orchestra.run.smithi013.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-16T20:20:55.686 INFO:teuthology.orchestra.run.smithi013.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-16T20:20:55.686 INFO:teuthology.orchestra.run.smithi013.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-16T20:20:55.686 INFO:teuthology.orchestra.run.smithi013.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-16T20:20:55.686 INFO:teuthology.orchestra.run.smithi013.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-16T20:20:55.686 INFO:teuthology.orchestra.run.smithi013.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-16T20:20:55.686 INFO:teuthology.orchestra.run.smithi013.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-16T20:20:55.686 INFO:teuthology.orchestra.run.smithi013.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-16T20:20:55.687 INFO:teuthology.orchestra.run.smithi013.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-16T20:20:55.687 INFO:teuthology.orchestra.run.smithi013.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-16T20:20:55.687 INFO:teuthology.orchestra.run.smithi013.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-16T20:20:55.687 INFO:teuthology.orchestra.run.smithi013.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-16T20:20:55.687 INFO:teuthology.orchestra.run.smithi013.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-16T20:20:55.688 INFO:teuthology.orchestra.run.smithi013.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-16T20:20:55.688 INFO:teuthology.orchestra.run.smithi013.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-16T20:20:55.699 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-16T20:20:55.780 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-16T20:20:55.784 INFO:teuthology.orchestra.run.smithi013.stdout:The following NEW packages will be installed: 2020-11-16T20:20:55.785 INFO:teuthology.orchestra.run.smithi013.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-16T20:20:55.785 INFO:teuthology.orchestra.run.smithi013.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-16T20:20:55.785 INFO:teuthology.orchestra.run.smithi013.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2020-11-16T20:20:55.786 INFO:teuthology.orchestra.run.smithi013.stdout: libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 liboath0 2020-11-16T20:20:55.786 INFO:teuthology.orchestra.run.smithi013.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2020-11-16T20:20:55.787 INFO:teuthology.orchestra.run.smithi013.stdout: libxslt1.1 nvme-cli python-attr python-funcsigs python-pastedeploy-tpl 2020-11-16T20:20:55.787 INFO:teuthology.orchestra.run.smithi013.stdout: python-pluggy python-py python-pytest python3-asn1crypto python3-attr 2020-11-16T20:20:55.787 INFO:teuthology.orchestra.run.smithi013.stdout: python3-bcrypt python3-bs4 python3-ceph-argparse python3-ceph-common 2020-11-16T20:20:55.787 INFO:teuthology.orchestra.run.smithi013.stdout: python3-cephfs python3-cffi-backend python3-cherrypy3 python3-cryptography 2020-11-16T20:20:55.787 INFO:teuthology.orchestra.run.smithi013.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-16T20:20:55.787 INFO:teuthology.orchestra.run.smithi013.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-16T20:20:55.788 INFO:teuthology.orchestra.run.smithi013.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-16T20:20:55.788 INFO:teuthology.orchestra.run.smithi013.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-16T20:20:55.788 INFO:teuthology.orchestra.run.smithi013.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-16T20:20:55.788 INFO:teuthology.orchestra.run.smithi013.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-16T20:20:55.789 INFO:teuthology.orchestra.run.smithi013.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-16T20:20:55.789 INFO:teuthology.orchestra.run.smithi013.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-16T20:20:55.789 INFO:teuthology.orchestra.run.smithi013.stdout: python3-pyinotify python3-pyparsing python3-pytest python3-rados python3-rbd 2020-11-16T20:20:55.789 INFO:teuthology.orchestra.run.smithi013.stdout: python3-repoze.lru python3-rgw python3-routes python3-rsa python3-scipy 2020-11-16T20:20:55.789 INFO:teuthology.orchestra.run.smithi013.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-16T20:20:55.789 INFO:teuthology.orchestra.run.smithi013.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-16T20:20:55.789 INFO:teuthology.orchestra.run.smithi013.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-16T20:20:55.790 INFO:teuthology.orchestra.run.smithi013.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-16T20:20:55.790 INFO:teuthology.orchestra.run.smithi013.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-16T20:20:55.790 INFO:teuthology.orchestra.run.smithi013.stdout:The following packages will be upgraded: 2020-11-16T20:20:55.790 INFO:teuthology.orchestra.run.smithi013.stdout: librados2 librbd1 2020-11-16T20:20:55.941 INFO:teuthology.orchestra.run.smithi013.stdout:2 upgraded, 110 newly installed, 0 to remove and 1 not upgraded. 2020-11-16T20:20:55.941 INFO:teuthology.orchestra.run.smithi013.stdout:Need to get 168 MB of archives. 2020-11-16T20:20:55.942 INFO:teuthology.orchestra.run.smithi013.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-16T20:20:55.942 INFO:teuthology.orchestra.run.smithi013.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7219-g911cc357-1bionic [1,921 kB] 2020-11-16T20:20:56.009 INFO:teuthology.orchestra.run.smithi013.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-16T20:20:56.040 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-16T20:20:56.088 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-16T20:20:56.089 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7219-g911cc357-1bionic [3,157 kB] 2020-11-16T20:20:56.090 INFO:teuthology.orchestra.run.smithi013.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-16T20:20:56.105 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-16T20:20:56.117 INFO:teuthology.orchestra.run.smithi013.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7219-g911cc357-1bionic [3,157 kB] 2020-11-16T20:20:56.202 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-16T20:20:56.205 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-16T20:20:56.232 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-16T20:20:56.234 INFO:teuthology.orchestra.run.smithi013.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7219-g911cc357-1bionic [479 kB] 2020-11-16T20:20:56.241 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-16T20:20:56.251 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-16T20:20:56.256 INFO:teuthology.orchestra.run.smithi013.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7219-g911cc357-1bionic [332 kB] 2020-11-16T20:20:56.261 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-16T20:20:56.269 INFO:teuthology.orchestra.run.smithi013.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7219-g911cc357-1bionic [20.6 kB] 2020-11-16T20:20:56.269 INFO:teuthology.orchestra.run.smithi013.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7219-g911cc357-1bionic [168 kB] 2020-11-16T20:20:56.269 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-16T20:20:56.275 INFO:teuthology.orchestra.run.smithi013.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7219-g911cc357-1bionic [27.7 kB] 2020-11-16T20:20:56.276 INFO:teuthology.orchestra.run.smithi013.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7219-g911cc357-1bionic [330 kB] 2020-11-16T20:20:56.293 INFO:teuthology.orchestra.run.smithi013.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7219-g911cc357-1bionic [3,177 kB] 2020-11-16T20:20:56.293 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-16T20:20:56.297 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-16T20:20:56.299 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-16T20:20:56.351 INFO:teuthology.orchestra.run.smithi013.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-16T20:20:56.360 INFO:teuthology.orchestra.run.smithi013.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-16T20:20:56.365 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-16T20:20:56.366 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-16T20:20:56.368 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-16T20:20:56.369 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-16T20:20:56.373 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-16T20:20:56.374 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-16T20:20:56.375 INFO:teuthology.orchestra.run.smithi013.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-16T20:20:56.426 INFO:teuthology.orchestra.run.smithi013.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7219-g911cc357-1bionic [99.1 kB] 2020-11-16T20:20:56.428 INFO:teuthology.orchestra.run.smithi013.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7219-g911cc357-1bionic [414 kB] 2020-11-16T20:20:56.443 INFO:teuthology.orchestra.run.smithi013.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7219-g911cc357-1bionic [20.5 MB] 2020-11-16T20:20:56.444 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-16T20:20:56.447 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-16T20:20:56.450 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-16T20:20:56.452 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-16T20:20:56.472 INFO:teuthology.orchestra.run.smithi013.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-16T20:20:56.475 INFO:teuthology.orchestra.run.smithi013.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-16T20:20:56.507 INFO:teuthology.orchestra.run.smithi013.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-16T20:20:56.515 INFO:teuthology.orchestra.run.smithi013.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-16T20:20:56.523 INFO:teuthology.orchestra.run.smithi013.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-16T20:20:56.524 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-16T20:20:56.533 INFO:teuthology.orchestra.run.smithi013.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-16T20:20:56.542 INFO:teuthology.orchestra.run.smithi013.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-16T20:20:56.543 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-16T20:20:56.543 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-16T20:20:56.544 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-16T20:20:56.545 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-16T20:20:56.549 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-16T20:20:56.562 INFO:teuthology.orchestra.run.smithi013.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-16T20:20:56.565 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7219-g911cc357-1bionic [479 kB] 2020-11-16T20:20:56.566 INFO:teuthology.orchestra.run.smithi013.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-16T20:20:56.594 INFO:teuthology.orchestra.run.smithi013.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-16T20:20:56.604 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-16T20:20:56.605 INFO:teuthology.orchestra.run.smithi013.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-16T20:20:56.606 INFO:teuthology.orchestra.run.smithi013.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-16T20:20:56.608 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-16T20:20:56.609 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-16T20:20:56.610 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-16T20:20:56.669 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7219-g911cc357-1bionic [332 kB] 2020-11-16T20:20:56.675 INFO:teuthology.orchestra.run.smithi013.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-16T20:20:56.675 INFO:teuthology.orchestra.run.smithi013.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-16T20:20:56.678 INFO:teuthology.orchestra.run.smithi013.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-16T20:20:56.678 INFO:teuthology.orchestra.run.smithi013.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-16T20:20:56.679 INFO:teuthology.orchestra.run.smithi013.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-16T20:20:56.681 INFO:teuthology.orchestra.run.smithi013.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-16T20:20:56.683 INFO:teuthology.orchestra.run.smithi013.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-16T20:20:56.705 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7219-g911cc357-1bionic [20.6 kB] 2020-11-16T20:20:56.706 INFO:teuthology.orchestra.run.smithi092.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7219-g911cc357-1bionic [168 kB] 2020-11-16T20:20:56.709 INFO:teuthology.orchestra.run.smithi092.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-16T20:20:56.709 INFO:teuthology.orchestra.run.smithi092.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-16T20:20:56.735 INFO:teuthology.orchestra.run.smithi092.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7219-g911cc357-1bionic [27.7 kB] 2020-11-16T20:20:56.736 INFO:teuthology.orchestra.run.smithi092.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7219-g911cc357-1bionic [330 kB] 2020-11-16T20:20:56.755 INFO:teuthology.orchestra.run.smithi013.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-16T20:20:56.756 INFO:teuthology.orchestra.run.smithi013.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-16T20:20:56.770 INFO:teuthology.orchestra.run.smithi013.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-16T20:20:56.798 INFO:teuthology.orchestra.run.smithi092.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7219-g911cc357-1bionic [3,177 kB] 2020-11-16T20:20:56.836 INFO:teuthology.orchestra.run.smithi013.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-16T20:20:56.837 INFO:teuthology.orchestra.run.smithi013.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-16T20:20:56.838 INFO:teuthology.orchestra.run.smithi013.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-16T20:20:56.842 INFO:teuthology.orchestra.run.smithi013.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-16T20:20:56.847 INFO:teuthology.orchestra.run.smithi013.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-16T20:20:56.848 INFO:teuthology.orchestra.run.smithi013.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-16T20:20:56.849 INFO:teuthology.orchestra.run.smithi013.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-16T20:20:56.916 INFO:teuthology.orchestra.run.smithi013.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-16T20:20:56.974 INFO:teuthology.orchestra.run.smithi092.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-16T20:20:57.004 INFO:teuthology.orchestra.run.smithi013.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-16T20:20:57.005 INFO:teuthology.orchestra.run.smithi013.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-16T20:20:57.007 INFO:teuthology.orchestra.run.smithi092.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-16T20:21:01.120 INFO:teuthology.orchestra.run.smithi013.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-16T20:21:01.121 INFO:teuthology.orchestra.run.smithi013.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-16T20:21:01.121 INFO:teuthology.orchestra.run.smithi013.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-16T20:21:01.121 INFO:teuthology.orchestra.run.smithi013.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7219-g911cc357-1bionic [5,566 kB] 2020-11-16T20:21:01.121 INFO:teuthology.orchestra.run.smithi013.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-16T20:21:01.121 INFO:teuthology.orchestra.run.smithi013.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-16T20:21:01.121 INFO:teuthology.orchestra.run.smithi013.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-16T20:21:01.122 INFO:teuthology.orchestra.run.smithi013.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-16T20:21:01.122 INFO:teuthology.orchestra.run.smithi013.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-16T20:21:01.122 INFO:teuthology.orchestra.run.smithi013.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-16T20:21:01.122 INFO:teuthology.orchestra.run.smithi013.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-16T20:21:01.122 INFO:teuthology.orchestra.run.smithi013.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-16T20:21:01.122 INFO:teuthology.orchestra.run.smithi013.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-16T20:21:01.122 INFO:teuthology.orchestra.run.smithi013.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-16T20:21:01.123 INFO:teuthology.orchestra.run.smithi013.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-16T20:21:01.123 INFO:teuthology.orchestra.run.smithi013.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-16T20:21:01.123 INFO:teuthology.orchestra.run.smithi013.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-16T20:21:01.123 INFO:teuthology.orchestra.run.smithi013.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-16T20:21:01.123 INFO:teuthology.orchestra.run.smithi013.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-16T20:21:01.123 INFO:teuthology.orchestra.run.smithi013.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-16T20:21:01.123 INFO:teuthology.orchestra.run.smithi013.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-16T20:21:01.124 INFO:teuthology.orchestra.run.smithi013.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7219-g911cc357-1bionic [165 kB] 2020-11-16T20:21:01.124 INFO:teuthology.orchestra.run.smithi013.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7219-g911cc357-1bionic [1,362 kB] 2020-11-16T20:21:01.124 INFO:teuthology.orchestra.run.smithi013.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7219-g911cc357-1bionic [6,657 kB] 2020-11-16T20:21:01.124 INFO:teuthology.orchestra.run.smithi013.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-16T20:21:01.124 INFO:teuthology.orchestra.run.smithi013.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-16T20:21:01.124 INFO:teuthology.orchestra.run.smithi013.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-16T20:21:01.124 INFO:teuthology.orchestra.run.smithi013.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-16T20:21:01.125 INFO:teuthology.orchestra.run.smithi013.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-16T20:21:01.125 INFO:teuthology.orchestra.run.smithi013.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-16T20:21:01.125 INFO:teuthology.orchestra.run.smithi013.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-16T20:21:01.125 INFO:teuthology.orchestra.run.smithi013.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-16T20:21:01.125 INFO:teuthology.orchestra.run.smithi013.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-16T20:21:01.125 INFO:teuthology.orchestra.run.smithi013.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-16T20:21:01.125 INFO:teuthology.orchestra.run.smithi013.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-16T20:21:01.125 INFO:teuthology.orchestra.run.smithi013.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-16T20:21:01.126 INFO:teuthology.orchestra.run.smithi013.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-16T20:21:01.126 INFO:teuthology.orchestra.run.smithi013.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-16T20:21:01.126 INFO:teuthology.orchestra.run.smithi013.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-16T20:21:01.126 INFO:teuthology.orchestra.run.smithi013.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-16T20:21:01.126 INFO:teuthology.orchestra.run.smithi013.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7219-g911cc357-1bionic [24.6 MB] 2020-11-16T20:21:01.126 INFO:teuthology.orchestra.run.smithi013.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-16T20:21:01.126 INFO:teuthology.orchestra.run.smithi013.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-16T20:21:01.127 INFO:teuthology.orchestra.run.smithi013.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-16T20:21:01.127 INFO:teuthology.orchestra.run.smithi013.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-16T20:21:01.127 INFO:teuthology.orchestra.run.smithi013.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-16T20:21:01.127 INFO:teuthology.orchestra.run.smithi013.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-16T20:21:01.127 INFO:teuthology.orchestra.run.smithi013.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-16T20:21:01.127 INFO:teuthology.orchestra.run.smithi013.stdout:Get:98 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-16T20:21:01.127 INFO:teuthology.orchestra.run.smithi013.stdout:Get:99 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-16T20:21:01.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7219-g911cc357-1bionic [3,804 B] 2020-11-16T20:21:01.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7219-g911cc357-1bionic [563 kB] 2020-11-16T20:21:01.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7219-g911cc357-1bionic [1,942 kB] 2020-11-16T20:21:01.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7219-g911cc357-1bionic [51.7 kB] 2020-11-16T20:21:01.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7219-g911cc357-1bionic [56.2 kB] 2020-11-16T20:21:01.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7219-g911cc357-1bionic [2,876 kB] 2020-11-16T20:21:01.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7219-g911cc357-1bionic [7,891 kB] 2020-11-16T20:21:01.128 INFO:teuthology.orchestra.run.smithi013.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7219-g911cc357-1bionic [3,924 B] 2020-11-16T20:21:01.129 INFO:teuthology.orchestra.run.smithi013.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7219-g911cc357-1bionic [22.3 kB] 2020-11-16T20:21:01.129 INFO:teuthology.orchestra.run.smithi013.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7219-g911cc357-1bionic [49.6 MB] 2020-11-16T20:21:01.129 INFO:teuthology.orchestra.run.smithi092.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-16T20:21:01.129 INFO:teuthology.orchestra.run.smithi092.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-16T20:21:01.129 INFO:teuthology.orchestra.run.smithi092.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-16T20:21:01.129 INFO:teuthology.orchestra.run.smithi092.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-16T20:21:01.130 INFO:teuthology.orchestra.run.smithi092.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-16T20:21:01.130 INFO:teuthology.orchestra.run.smithi092.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-16T20:21:01.130 INFO:teuthology.orchestra.run.smithi092.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-16T20:21:01.130 INFO:teuthology.orchestra.run.smithi092.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-16T20:21:01.130 INFO:teuthology.orchestra.run.smithi092.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-16T20:21:01.130 INFO:teuthology.orchestra.run.smithi092.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-16T20:21:01.130 INFO:teuthology.orchestra.run.smithi092.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-16T20:21:01.131 INFO:teuthology.orchestra.run.smithi092.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-16T20:21:01.131 INFO:teuthology.orchestra.run.smithi092.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-16T20:21:01.131 INFO:teuthology.orchestra.run.smithi092.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-16T20:21:01.131 INFO:teuthology.orchestra.run.smithi092.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-16T20:21:01.131 INFO:teuthology.orchestra.run.smithi092.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-16T20:21:01.131 INFO:teuthology.orchestra.run.smithi092.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-16T20:21:01.131 INFO:teuthology.orchestra.run.smithi092.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-16T20:21:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-16T20:21:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-16T20:21:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-16T20:21:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-16T20:21:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-16T20:21:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-16T20:21:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-16T20:21:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-16T20:21:01.133 INFO:teuthology.orchestra.run.smithi092.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-16T20:21:01.133 INFO:teuthology.orchestra.run.smithi092.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-16T20:21:01.133 INFO:teuthology.orchestra.run.smithi092.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-16T20:21:01.133 INFO:teuthology.orchestra.run.smithi092.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-16T20:21:01.134 INFO:teuthology.orchestra.run.smithi092.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-16T20:21:01.134 INFO:teuthology.orchestra.run.smithi092.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7219-g911cc357-1bionic [99.1 kB] 2020-11-16T20:21:01.134 INFO:teuthology.orchestra.run.smithi092.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7219-g911cc357-1bionic [414 kB] 2020-11-16T20:21:01.135 INFO:teuthology.orchestra.run.smithi092.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7219-g911cc357-1bionic [20.5 MB] 2020-11-16T20:21:01.135 INFO:teuthology.orchestra.run.smithi092.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-16T20:21:01.135 INFO:teuthology.orchestra.run.smithi092.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-16T20:21:01.135 INFO:teuthology.orchestra.run.smithi092.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-16T20:21:01.135 INFO:teuthology.orchestra.run.smithi092.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-16T20:21:01.135 INFO:teuthology.orchestra.run.smithi092.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-16T20:21:01.135 INFO:teuthology.orchestra.run.smithi092.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-16T20:21:01.136 INFO:teuthology.orchestra.run.smithi092.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-16T20:21:01.136 INFO:teuthology.orchestra.run.smithi092.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-16T20:21:01.136 INFO:teuthology.orchestra.run.smithi092.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-16T20:21:01.136 INFO:teuthology.orchestra.run.smithi092.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-16T20:21:01.136 INFO:teuthology.orchestra.run.smithi092.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-16T20:21:01.136 INFO:teuthology.orchestra.run.smithi092.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-16T20:21:01.136 INFO:teuthology.orchestra.run.smithi092.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7219-g911cc357-1bionic [5,566 kB] 2020-11-16T20:21:01.738 INFO:teuthology.orchestra.run.smithi092.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7219-g911cc357-1bionic [165 kB] 2020-11-16T20:21:01.787 INFO:teuthology.orchestra.run.smithi092.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7219-g911cc357-1bionic [1,362 kB] 2020-11-16T20:21:02.029 INFO:teuthology.orchestra.run.smithi092.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7219-g911cc357-1bionic [6,657 kB] 2020-11-16T20:21:02.158 INFO:teuthology.orchestra.run.smithi013.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7219-g911cc357-1bionic [19.5 kB] 2020-11-16T20:21:02.158 INFO:teuthology.orchestra.run.smithi013.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7219-g911cc357-1bionic [9,855 kB] 2020-11-16T20:21:02.680 INFO:teuthology.orchestra.run.smithi013.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7219-g911cc357-1bionic [77.0 kB] 2020-11-16T20:21:02.894 INFO:teuthology.orchestra.run.smithi092.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7219-g911cc357-1bionic [24.6 MB] 2020-11-16T20:21:03.416 INFO:teuthology.orchestra.run.smithi013.stdout:Fetched 168 MB in 7s (24.5 MB/s) 2020-11-16T20:21:03.562 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-16T20:21:03.614 INFO:teuthology.orchestra.run.smithi013.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137607 files and directories currently installed.) 2020-11-16T20:21:03.618 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-16T20:21:03.660 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T20:21:03.988 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-16T20:21:04.004 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../001-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-16T20:21:04.089 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T20:21:04.464 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../002-librbd1_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:04.584 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking librbd1 (16.0.0-7219-g911cc357-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T20:21:04.671 INFO:teuthology.orchestra.run.smithi092.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7219-g911cc357-1bionic [3,804 B] 2020-11-16T20:21:04.672 INFO:teuthology.orchestra.run.smithi092.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7219-g911cc357-1bionic [563 kB] 2020-11-16T20:21:04.701 INFO:teuthology.orchestra.run.smithi092.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7219-g911cc357-1bionic [1,942 kB] 2020-11-16T20:21:04.845 INFO:teuthology.orchestra.run.smithi092.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7219-g911cc357-1bionic [51.7 kB] 2020-11-16T20:21:04.846 INFO:teuthology.orchestra.run.smithi092.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7219-g911cc357-1bionic [56.2 kB] 2020-11-16T20:21:04.848 INFO:teuthology.orchestra.run.smithi092.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7219-g911cc357-1bionic [2,876 kB] 2020-11-16T20:21:05.056 INFO:teuthology.orchestra.run.smithi092.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7219-g911cc357-1bionic [7,891 kB] 2020-11-16T20:21:05.120 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../003-librados2_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:05.255 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking librados2 (16.0.0-7219-g911cc357-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T20:21:05.604 INFO:teuthology.orchestra.run.smithi092.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7219-g911cc357-1bionic [3,924 B] 2020-11-16T20:21:05.605 INFO:teuthology.orchestra.run.smithi092.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7219-g911cc357-1bionic [22.3 kB] 2020-11-16T20:21:05.609 INFO:teuthology.orchestra.run.smithi092.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7219-g911cc357-1bionic [49.6 MB] 2020-11-16T20:21:05.900 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package libcephfs2. 2020-11-16T20:21:05.915 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../004-libcephfs2_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:05.959 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking libcephfs2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:06.411 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-rados. 2020-11-16T20:21:06.425 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../005-python3-rados_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:06.469 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-rados (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:06.755 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-16T20:21:06.770 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../006-python3-ceph-argparse_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:06.806 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-ceph-argparse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:07.083 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-cephfs. 2020-11-16T20:21:07.096 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../007-python3-cephfs_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:07.133 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-cephfs (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:07.444 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-16T20:21:07.460 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../008-python3-ceph-common_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:07.502 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-ceph-common (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:07.855 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-prettytable. 2020-11-16T20:21:07.872 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../009-python3-prettytable_0.7.2-3_all.deb ... 2020-11-16T20:21:07.914 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-16T20:21:08.141 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-rbd. 2020-11-16T20:21:08.157 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../010-python3-rbd_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:08.191 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-rbd (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:08.552 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-16T20:21:08.567 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../011-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-16T20:21:08.610 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T20:21:08.930 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-16T20:21:08.944 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../012-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-16T20:21:08.988 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T20:21:09.333 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package librgw2. 2020-11-16T20:21:09.348 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../013-librgw2_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:09.382 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking librgw2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:09.893 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-rgw. 2020-11-16T20:21:09.907 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../014-python3-rgw_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:09.943 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-rgw (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:09.965 INFO:teuthology.orchestra.run.smithi092.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7219-g911cc357-1bionic [19.5 kB] 2020-11-16T20:21:09.965 INFO:teuthology.orchestra.run.smithi092.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7219-g911cc357-1bionic [9,855 kB] 2020-11-16T20:21:10.246 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-16T20:21:10.259 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../015-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-16T20:21:10.304 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T20:21:10.615 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-16T20:21:10.629 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../016-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-16T20:21:10.666 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T20:21:10.898 INFO:teuthology.orchestra.run.smithi092.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-kefu-testing-2020-11-16-1845/911cc35794d871071a8bebfb0919482263b1d18b/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7219-g911cc357-1bionic [77.0 kB] 2020-11-16T20:21:10.985 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package liboath0. 2020-11-16T20:21:10.999 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../017-liboath0_2.6.1-1_amd64.deb ... 2020-11-16T20:21:11.035 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-16T20:21:11.337 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package libradosstriper1. 2020-11-16T20:21:11.351 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../018-libradosstriper1_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:11.421 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking libradosstriper1 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:11.615 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 168 MB in 15s (10.9 MB/s) 2020-11-16T20:21:11.770 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-16T20:21:11.818 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 137607 files and directories currently installed.) 2020-11-16T20:21:11.822 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-16T20:21:11.860 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T20:21:11.916 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-common. 2020-11-16T20:21:11.929 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../019-ceph-common_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:11.966 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-common (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:12.204 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-16T20:21:12.218 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../001-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-16T20:21:12.305 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T20:21:12.646 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../002-librbd1_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:12.759 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librbd1 (16.0.0-7219-g911cc357-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T20:21:13.326 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../003-librados2_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:13.455 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librados2 (16.0.0-7219-g911cc357-1bionic) over (12.2.13-0ubuntu0.18.04.4) ... 2020-11-16T20:21:13.829 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-base. 2020-11-16T20:21:13.843 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../020-ceph-base_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:13.896 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-base (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:14.083 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs2. 2020-11-16T20:21:14.096 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../004-libcephfs2_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:14.133 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:14.403 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rados. 2020-11-16T20:21:14.417 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../005-python3-rados_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:14.494 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rados (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:14.838 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-16T20:21:14.851 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../006-python3-ceph-argparse_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:14.867 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-dateutil. 2020-11-16T20:21:14.881 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../021-python3-dateutil_2.6.1-1_all.deb ... 2020-11-16T20:21:14.888 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-argparse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:14.925 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-16T20:21:15.183 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cephfs. 2020-11-16T20:21:15.198 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../007-python3-cephfs_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:15.241 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cephfs (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:15.303 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-16T20:21:15.319 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../022-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-16T20:21:15.361 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-16T20:21:15.511 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-16T20:21:15.526 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../008-python3-ceph-common_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:15.561 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-common (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:15.597 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-16T20:21:15.611 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../023-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-16T20:21:15.656 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-16T20:21:15.914 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-prettytable. 2020-11-16T20:21:15.928 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../009-python3-prettytable_0.7.2-3_all.deb ... 2020-11-16T20:21:15.972 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-16T20:21:15.991 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-cryptography. 2020-11-16T20:21:16.008 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../024-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-16T20:21:16.050 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T20:21:16.216 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rbd. 2020-11-16T20:21:16.230 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../010-python3-rbd_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:16.275 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rbd (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:16.470 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-openssl. 2020-11-16T20:21:16.485 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../025-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-16T20:21:16.528 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T20:21:16.611 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-16T20:21:16.624 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../011-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-16T20:21:16.653 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T20:21:16.806 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-16T20:21:16.823 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:16.865 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:17.006 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-16T20:21:17.019 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../012-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-16T20:21:17.056 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T20:21:17.219 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-16T20:21:17.232 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-16T20:21:17.269 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-16T20:21:17.400 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librgw2. 2020-11-16T20:21:17.414 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../013-librgw2_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:17.458 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librgw2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:17.663 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-16T20:21:17.680 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-16T20:21:17.713 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-16T20:21:18.028 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rgw. 2020-11-16T20:21:18.041 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../014-python3-rgw_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:18.050 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-jwt. 2020-11-16T20:21:18.063 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../029-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-16T20:21:18.086 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rgw (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:18.100 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-16T20:21:18.355 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-16T20:21:18.371 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../015-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-16T20:21:18.394 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-logutils. 2020-11-16T20:21:18.410 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-5_all.deb ... 2020-11-16T20:21:18.413 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T20:21:18.453 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-16T20:21:18.725 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-16T20:21:18.738 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../016-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-16T20:21:18.764 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-16T20:21:18.780 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../031-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-16T20:21:18.783 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T20:21:18.822 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-16T20:21:19.108 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-mako. 2020-11-16T20:21:19.124 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../032-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-16T20:21:19.166 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-16T20:21:19.184 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liboath0. 2020-11-16T20:21:19.194 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../017-liboath0_2.6.1-1_amd64.deb ... 2020-11-16T20:21:19.236 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-16T20:21:19.461 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-16T20:21:19.475 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-16T20:21:19.480 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libradosstriper1. 2020-11-16T20:21:19.496 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../018-libradosstriper1_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:19.519 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-16T20:21:19.530 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libradosstriper1 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:19.789 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-16T20:21:19.803 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-16T20:21:19.847 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-16T20:21:20.008 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-common. 2020-11-16T20:21:20.022 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../019-ceph-common_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:20.067 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-common (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:20.152 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-webob. 2020-11-16T20:21:20.166 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../035-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-16T20:21:20.210 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T20:21:20.513 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-bs4. 2020-11-16T20:21:20.527 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../036-python3-bs4_4.6.0-1_all.deb ... 2020-11-16T20:21:20.563 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-16T20:21:20.874 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-waitress. 2020-11-16T20:21:20.888 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../037-python3-waitress_1.0.1-1_all.deb ... 2020-11-16T20:21:20.932 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-16T20:21:21.261 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-tempita. 2020-11-16T20:21:21.275 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../038-python3-tempita_0.5.2-2_all.deb ... 2020-11-16T20:21:21.319 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-16T20:21:21.613 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-paste. 2020-11-16T20:21:21.627 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../039-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-16T20:21:21.671 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T20:21:21.897 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-base. 2020-11-16T20:21:21.912 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../020-ceph-base_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:21.957 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-base (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:21.991 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-16T20:21:22.007 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../040-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-16T20:21:22.058 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T20:21:22.386 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-16T20:21:22.400 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../041-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-16T20:21:22.461 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-16T20:21:22.772 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-webtest. 2020-11-16T20:21:22.787 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../042-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-16T20:21:22.830 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T20:21:22.952 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-dateutil. 2020-11-16T20:21:22.967 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../021-python3-dateutil_2.6.1-1_all.deb ... 2020-11-16T20:21:23.010 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-16T20:21:23.166 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pecan. 2020-11-16T20:21:23.180 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../043-python3-pecan_1.2.1-2_all.deb ... 2020-11-16T20:21:23.217 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-16T20:21:23.338 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-16T20:21:23.354 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../022-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-16T20:21:23.396 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-16T20:21:23.571 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-16T20:21:23.586 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../044-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-16T20:21:23.629 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T20:21:23.774 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-16T20:21:23.788 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../023-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-16T20:21:23.824 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-16T20:21:23.999 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mgr. 2020-11-16T20:21:24.016 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../045-ceph-mgr_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:24.057 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mgr (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:24.193 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cryptography. 2020-11-16T20:21:24.208 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../024-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-16T20:21:24.252 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T20:21:24.527 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mon. 2020-11-16T20:21:24.541 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../046-ceph-mon_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:24.577 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mon (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:24.622 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-openssl. 2020-11-16T20:21:24.637 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../025-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-16T20:21:24.680 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T20:21:24.933 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-16T20:21:24.949 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:25.000 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:25.363 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-16T20:21:25.378 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-16T20:21:25.421 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-16T20:21:25.539 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-osd. 2020-11-16T20:21:25.554 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../047-ceph-osd_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:25.597 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-osd (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:25.715 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-16T20:21:25.729 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-16T20:21:25.774 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-16T20:21:26.102 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jwt. 2020-11-16T20:21:26.117 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../029-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-16T20:21:26.160 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-16T20:21:26.514 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-logutils. 2020-11-16T20:21:26.528 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-5_all.deb ... 2020-11-16T20:21:26.572 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-16T20:21:26.866 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-16T20:21:26.881 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../031-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-16T20:21:26.933 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-16T20:21:27.221 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-mako. 2020-11-16T20:21:27.235 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../032-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-16T20:21:27.278 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-16T20:21:27.622 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-16T20:21:27.637 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-16T20:21:27.681 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-16T20:21:28.000 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-16T20:21:28.013 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-16T20:21:28.075 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-16T20:21:28.136 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph. 2020-11-16T20:21:28.152 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../048-ceph_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:28.203 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:28.378 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webob. 2020-11-16T20:21:28.393 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../035-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-16T20:21:28.436 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T20:21:28.531 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-fuse. 2020-11-16T20:21:28.544 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../049-ceph-fuse_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:28.581 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-fuse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:28.976 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mds. 2020-11-16T20:21:28.989 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../050-ceph-mds_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:29.026 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mds (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:29.545 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package cephadm. 2020-11-16T20:21:29.559 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../051-cephadm_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:29.603 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking cephadm (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:29.747 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-bs4. 2020-11-16T20:21:29.761 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../036-python3-bs4_4.6.0-1_all.deb ... 2020-11-16T20:21:29.805 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-16T20:21:29.940 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-jinja2. 2020-11-16T20:21:29.954 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../052-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-16T20:21:29.998 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T20:21:30.167 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-waitress. 2020-11-16T20:21:30.181 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../037-python3-waitress_1.0.1-1_all.deb ... 2020-11-16T20:21:30.225 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-16T20:21:30.326 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-16T20:21:30.340 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../053-ceph-mgr-cephadm_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:30.384 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:30.553 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tempita. 2020-11-16T20:21:30.569 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../038-python3-tempita_0.5.2-2_all.deb ... 2020-11-16T20:21:30.612 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-16T20:21:30.679 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-16T20:21:30.696 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../054-python3-repoze.lru_0.7-1_all.deb ... 2020-11-16T20:21:30.737 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-16T20:21:30.923 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-paste. 2020-11-16T20:21:30.937 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../039-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-16T20:21:30.981 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T20:21:31.015 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-routes. 2020-11-16T20:21:31.031 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../055-python3-routes_2.4.1-1_all.deb ... 2020-11-16T20:21:31.073 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-16T20:21:31.319 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-16T20:21:31.333 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../040-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-16T20:21:31.385 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T20:21:31.401 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-16T20:21:31.415 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../056-ceph-mgr-dashboard_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:31.459 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:31.687 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-16T20:21:31.703 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../041-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-16T20:21:31.746 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-16T20:21:32.023 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webtest. 2020-11-16T20:21:32.039 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../042-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-16T20:21:32.082 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T20:21:32.343 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-numpy. 2020-11-16T20:21:32.358 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../057-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-16T20:21:32.401 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T20:21:32.443 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pecan. 2020-11-16T20:21:32.459 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../043-python3-pecan_1.2.1-2_all.deb ... 2020-11-16T20:21:32.502 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-16T20:21:32.823 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-16T20:21:32.839 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../044-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-16T20:21:32.881 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T20:21:33.023 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-decorator. 2020-11-16T20:21:33.040 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../058-python3-decorator_4.1.2-1_all.deb ... 2020-11-16T20:21:33.082 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-16T20:21:33.262 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr. 2020-11-16T20:21:33.276 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../045-ceph-mgr_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:33.320 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:33.418 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-scipy. 2020-11-16T20:21:33.432 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../059-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-16T20:21:33.476 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T20:21:33.806 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mon. 2020-11-16T20:21:37.940 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-16T20:21:37.940 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../060-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-16T20:21:37.941 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-16T20:21:37.941 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-joblib. 2020-11-16T20:21:37.941 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../061-python3-joblib_0.11-1_all.deb ... 2020-11-16T20:21:37.941 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-16T20:21:37.941 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-sklearn. 2020-11-16T20:21:37.941 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../062-python3-sklearn_0.19.1-3_all.deb ... 2020-11-16T20:21:37.942 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-16T20:21:37.942 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-16T20:21:37.942 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../063-ceph-mgr-diskprediction-local_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:37.942 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:37.942 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-monotonic. 2020-11-16T20:21:37.942 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../064-python3-monotonic_1.1-2_all.deb ... 2020-11-16T20:21:37.942 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-16T20:21:37.943 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../046-ceph-mon_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:37.943 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mon (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:37.943 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-osd. 2020-11-16T20:21:37.943 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../047-ceph-osd_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:37.943 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-osd (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:37.943 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph. 2020-11-16T20:21:37.943 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../048-ceph_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:37.944 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:37.944 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-fuse. 2020-11-16T20:21:37.944 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../049-ceph-fuse_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:37.944 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-fuse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:37.944 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mds. 2020-11-16T20:21:37.944 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../050-ceph-mds_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:37.944 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mds (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:38.221 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-fasteners. 2020-11-16T20:21:38.235 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../065-python3-fasteners_0.12.0-3_all.deb ... 2020-11-16T20:21:38.279 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-16T20:21:38.315 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package cephadm. 2020-11-16T20:21:38.331 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../051-cephadm_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:38.373 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking cephadm (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:38.599 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-httplib2. 2020-11-16T20:21:38.614 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../066-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-16T20:21:38.657 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T20:21:38.692 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jinja2. 2020-11-16T20:21:38.706 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../052-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-16T20:21:38.750 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T20:21:38.935 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-16T20:21:38.952 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-16T20:21:38.993 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-16T20:21:39.095 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-16T20:21:39.109 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../053-ceph-mgr-cephadm_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:39.153 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:39.330 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-16T20:21:39.346 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-16T20:21:39.396 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T20:21:39.501 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-16T20:21:39.517 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../054-python3-repoze.lru_0.7-1_all.deb ... 2020-11-16T20:21:39.564 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-16T20:21:39.749 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-rsa. 2020-11-16T20:21:39.764 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../069-python3-rsa_3.4.2-1_all.deb ... 2020-11-16T20:21:39.816 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-16T20:21:39.884 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-routes. 2020-11-16T20:21:39.897 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../055-python3-routes_2.4.1-1_all.deb ... 2020-11-16T20:21:39.942 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-16T20:21:40.161 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-16T20:21:40.175 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../070-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-16T20:21:40.219 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T20:21:40.253 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-16T20:21:40.269 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../056-ceph-mgr-dashboard_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:40.311 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:40.707 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-16T20:21:40.722 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../071-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-16T20:21:40.765 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-16T20:21:41.110 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-setuptools. 2020-11-16T20:21:41.124 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../072-python3-setuptools_39.0.1-2_all.deb ... 2020-11-16T20:21:41.168 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-16T20:21:41.219 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-numpy. 2020-11-16T20:21:41.234 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../057-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-16T20:21:41.277 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T20:21:41.563 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-websocket. 2020-11-16T20:21:41.577 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../073-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-16T20:21:41.613 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T20:21:41.849 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-decorator. 2020-11-16T20:21:41.864 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../058-python3-decorator_4.1.2-1_all.deb ... 2020-11-16T20:21:41.907 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-16T20:21:41.991 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-16T20:21:42.007 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../074-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-16T20:21:42.058 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T20:21:42.244 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-scipy. 2020-11-16T20:21:42.260 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../059-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-16T20:21:42.302 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T20:21:42.487 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-16T20:21:42.504 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../075-ceph-mgr-k8sevents_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:42.545 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:42.865 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-16T20:21:42.879 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../076-python3-json-pointer_1.10-1_all.deb ... 2020-11-16T20:21:42.923 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-16T20:21:43.326 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-16T20:21:43.340 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../077-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-16T20:21:43.396 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T20:21:43.694 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-16T20:21:43.709 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../060-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-16T20:21:43.729 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-16T20:21:43.744 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../078-ceph-mgr-rook_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:43.752 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-16T20:21:43.787 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-mgr-rook (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:44.115 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-16T20:21:44.131 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../079-libonig4_6.7.0-1_amd64.deb ... 2020-11-16T20:21:44.174 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-16T20:21:44.339 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-joblib. 2020-11-16T20:21:44.353 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../061-python3-joblib_0.11-1_all.deb ... 2020-11-16T20:21:44.397 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-16T20:21:44.526 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-16T20:21:44.540 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../080-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-16T20:21:44.688 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T20:21:44.742 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn. 2020-11-16T20:21:44.758 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../062-python3-sklearn_0.19.1-3_all.deb ... 2020-11-16T20:21:44.800 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-16T20:21:44.940 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package jq. 2020-11-16T20:21:44.955 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../081-jq_1.5+dfsg-2_amd64.deb ... 2020-11-16T20:21:44.999 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-16T20:21:45.259 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package socat. 2020-11-16T20:21:45.273 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../082-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-16T20:21:45.318 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-16T20:21:45.339 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-16T20:21:45.353 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../063-ceph-mgr-diskprediction-local_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:45.406 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:45.813 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-16T20:21:45.829 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../083-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-16T20:21:45.879 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T20:21:46.141 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package xmlstarlet. 2020-11-16T20:21:46.155 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../084-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-16T20:21:46.199 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-16T20:21:46.509 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-monotonic. 2020-11-16T20:21:46.525 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../064-python3-monotonic_1.1-2_all.deb ... 2020-11-16T20:21:46.553 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package ceph-test. 2020-11-16T20:21:46.567 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-16T20:21:46.569 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../085-ceph-test_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:46.611 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking ceph-test (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:46.862 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-fasteners. 2020-11-16T20:21:46.876 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../065-python3-fasteners_0.12.0-3_all.deb ... 2020-11-16T20:21:46.928 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-16T20:21:47.231 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-httplib2. 2020-11-16T20:21:47.245 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../066-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-16T20:21:47.289 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T20:21:47.609 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-16T20:21:47.625 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-16T20:21:47.667 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-16T20:21:48.004 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-16T20:21:48.017 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-16T20:21:48.095 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T20:21:48.390 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rsa. 2020-11-16T20:21:48.406 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../069-python3-rsa_3.4.2-1_all.deb ... 2020-11-16T20:21:48.448 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-16T20:21:48.784 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-16T20:21:48.801 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../070-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-16T20:21:48.842 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T20:21:49.305 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-16T20:21:49.320 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../071-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-16T20:21:49.363 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-16T20:21:49.699 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-setuptools. 2020-11-16T20:21:49.713 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../072-python3-setuptools_39.0.1-2_all.deb ... 2020-11-16T20:21:49.757 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-16T20:21:50.153 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-websocket. 2020-11-16T20:21:50.166 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../073-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-16T20:21:50.211 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T20:21:50.308 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-16T20:21:50.325 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../086-libcephfs-dev_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:50.366 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking libcephfs-dev (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:50.547 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-16T20:21:50.563 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../074-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-16T20:21:50.606 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T20:21:50.627 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package nvme-cli. 2020-11-16T20:21:50.642 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../087-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-16T20:21:50.685 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-16T20:21:51.034 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-16T20:21:51.048 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../075-ceph-mgr-k8sevents_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:51.093 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:51.223 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python-attr. 2020-11-16T20:21:51.238 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../088-python-attr_17.4.0-2_all.deb ... 2020-11-16T20:21:51.289 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-16T20:21:51.420 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-16T20:21:51.436 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../076-python3-json-pointer_1.10-1_all.deb ... 2020-11-16T20:21:51.478 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-16T20:21:51.601 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python-funcsigs. 2020-11-16T20:21:51.615 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../089-python-funcsigs_1.0.2-4_all.deb ... 2020-11-16T20:21:51.659 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-16T20:21:51.857 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-16T20:21:51.872 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../077-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-16T20:21:51.918 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T20:21:51.962 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python-pluggy. 2020-11-16T20:21:51.978 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../090-python-pluggy_0.6.0-1_all.deb ... 2020-11-16T20:21:52.020 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-16T20:21:52.209 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-16T20:21:52.226 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../078-ceph-mgr-rook_16.0.0-7219-g911cc357-1bionic_all.deb ... 2020-11-16T20:21:52.267 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-rook (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:52.323 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python-py. 2020-11-16T20:21:52.338 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../091-python-py_1.5.2-1_all.deb ... 2020-11-16T20:21:52.381 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-16T20:21:52.604 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-16T20:21:52.621 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../079-libonig4_6.7.0-1_amd64.deb ... 2020-11-16T20:21:52.662 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-16T20:21:52.734 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python-pytest. 2020-11-16T20:21:52.750 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../092-python-pytest_3.3.2-2_all.deb ... 2020-11-16T20:21:52.801 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-16T20:21:53.040 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-16T20:21:53.054 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../080-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-16T20:21:53.146 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-attr. 2020-11-16T20:21:53.162 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../093-python3-attr_17.4.0-2_all.deb ... 2020-11-16T20:21:53.175 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T20:21:53.205 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-16T20:21:53.461 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package jq. 2020-11-16T20:21:53.475 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../081-jq_1.5+dfsg-2_amd64.deb ... 2020-11-16T20:21:53.519 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-16T20:21:53.540 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-cycler. 2020-11-16T20:21:53.555 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../094-python3-cycler_0.10.0-1_all.deb ... 2020-11-16T20:21:53.598 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-16T20:21:53.772 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package socat. 2020-11-16T20:21:53.787 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../082-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-16T20:21:53.830 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-16T20:21:53.901 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-webencodings. 2020-11-16T20:21:53.916 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../095-python3-webencodings_0.5-2_all.deb ... 2020-11-16T20:21:53.960 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-16T20:21:54.254 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-html5lib. 2020-11-16T20:21:54.259 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-16T20:21:54.272 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../096-python3-html5lib_0.999999999-1_all.deb ... 2020-11-16T20:21:54.273 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../083-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-16T20:21:54.313 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-16T20:21:54.317 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T20:21:54.595 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package xmlstarlet. 2020-11-16T20:21:54.611 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../084-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-16T20:21:54.653 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-16T20:21:54.682 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-16T20:21:54.698 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../097-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-16T20:21:54.740 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T20:21:54.991 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-test. 2020-11-16T20:21:55.007 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../085-ceph-test_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:55.049 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-test (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:55.186 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-16T20:21:55.202 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../098-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-16T20:21:55.244 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T20:21:55.605 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-tz. 2020-11-16T20:21:55.619 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../099-python3-tz_2018.3-2_all.deb ... 2020-11-16T20:21:55.663 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-16T20:21:55.966 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-16T20:21:55.982 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../100-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-16T20:21:56.024 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T20:21:56.765 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-olefile. 2020-11-16T20:21:56.779 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../101-python3-olefile_0.45.1-1_all.deb ... 2020-11-16T20:21:56.822 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-16T20:21:57.142 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pastescript. 2020-11-16T20:21:57.158 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../102-python3-pastescript_2.0.2-2_all.deb ... 2020-11-16T20:21:57.200 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-16T20:21:57.554 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-16T20:21:57.568 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../103-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-16T20:21:57.612 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T20:21:57.999 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pluggy. 2020-11-16T20:21:58.015 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../104-python3-pluggy_0.6.0-1_all.deb ... 2020-11-16T20:21:58.074 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-16T20:21:58.376 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-py. 2020-11-16T20:21:58.391 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../105-python3-py_1.5.2-1_all.deb ... 2020-11-16T20:21:58.435 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-16T20:21:58.703 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-16T20:21:58.711 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../086-libcephfs-dev_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:21:58.753 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs-dev (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:21:58.755 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-16T20:21:58.770 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../106-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-16T20:21:58.822 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-16T20:21:59.038 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package nvme-cli. 2020-11-16T20:21:59.052 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../087-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-16T20:21:59.097 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-16T20:21:59.107 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-pytest. 2020-11-16T20:21:59.124 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../107-python3-pytest_3.3.2-2_all.deb ... 2020-11-16T20:21:59.166 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-16T20:21:59.502 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-simplejson. 2020-11-16T20:21:59.520 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../108-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-16T20:21:59.561 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-16T20:21:59.834 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-attr. 2020-11-16T20:21:59.849 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../088-python-attr_17.4.0-2_all.deb ... 2020-11-16T20:21:59.893 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-16T20:21:59.939 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-16T20:21:59.955 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../109-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-16T20:21:59.997 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T20:22:00.229 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-funcsigs. 2020-11-16T20:22:00.244 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../089-python-funcsigs_1.0.2-4_all.deb ... 2020-11-16T20:22:00.295 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-16T20:22:00.443 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package radosgw. 2020-11-16T20:22:00.458 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../110-radosgw_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:22:00.501 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking radosgw (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:00.589 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pluggy. 2020-11-16T20:22:00.606 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../090-python-pluggy_0.6.0-1_all.deb ... 2020-11-16T20:22:00.647 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-16T20:22:00.950 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-py. 2020-11-16T20:22:00.965 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../091-python-py_1.5.2-1_all.deb ... 2020-11-16T20:22:01.008 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-16T20:22:01.370 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pytest. 2020-11-16T20:22:01.386 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../092-python-pytest_3.3.2-2_all.deb ... 2020-11-16T20:22:01.428 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-16T20:22:01.548 INFO:teuthology.orchestra.run.smithi013.stdout:Selecting previously unselected package rbd-fuse. 2020-11-16T20:22:01.563 INFO:teuthology.orchestra.run.smithi013.stdout:Preparing to unpack .../111-rbd-fuse_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:22:01.631 INFO:teuthology.orchestra.run.smithi013.stdout:Unpacking rbd-fuse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:01.731 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-attr. 2020-11-16T20:22:01.746 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../093-python3-attr_17.4.0-2_all.deb ... 2020-11-16T20:22:01.781 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-16T20:22:01.943 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-16T20:22:02.067 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cycler. 2020-11-16T20:22:02.083 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../094-python3-cycler_0.10.0-1_all.deb ... 2020-11-16T20:22:02.125 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-16T20:22:02.311 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T20:22:02.453 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webencodings. 2020-11-16T20:22:02.468 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../095-python3-webencodings_0.5-2_all.deb ... 2020-11-16T20:22:02.511 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-16T20:22:02.695 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T20:22:02.797 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-html5lib. 2020-11-16T20:22:02.812 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../096-python3-html5lib_0.999999999-1_all.deb ... 2020-11-16T20:22:02.821 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T20:22:02.847 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-16T20:22:03.150 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-16T20:22:03.165 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../097-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-16T20:22:03.200 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T20:22:03.449 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T20:22:03.637 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-16T20:22:03.654 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../098-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-16T20:22:03.695 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T20:22:03.850 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-16T20:22:03.989 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tz. 2020-11-16T20:22:04.006 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../099-python3-tz_2018.3-2_all.deb ... 2020-11-16T20:22:04.047 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-16T20:22:04.135 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-16T20:22:04.350 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-16T20:22:04.365 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../100-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-16T20:22:04.428 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-16T20:22:04.471 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T20:22:04.637 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-16T20:22:04.772 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-16T20:22:05.115 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-16T20:22:05.156 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-olefile. 2020-11-16T20:22:05.171 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../101-python3-olefile_0.45.1-1_all.deb ... 2020-11-16T20:22:05.205 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-16T20:22:05.458 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastescript. 2020-11-16T20:22:05.472 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../102-python3-pastescript_2.0.2-2_all.deb ... 2020-11-16T20:22:05.484 INFO:teuthology.orchestra.run.smithi013.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-16T20:22:05.516 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-16T20:22:05.759 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-16T20:22:05.904 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-16T20:22:05.913 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../103-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-16T20:22:05.987 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T20:22:06.060 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-ceph-argparse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:06.356 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pluggy. 2020-11-16T20:22:06.370 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../104-python3-pluggy_0.6.0-1_all.deb ... 2020-11-16T20:22:06.386 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-16T20:22:06.415 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-16T20:22:06.688 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-16T20:22:06.776 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-py. 2020-11-16T20:22:06.792 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../105-python3-py_1.5.2-1_all.deb ... 2020-11-16T20:22:06.814 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T20:22:06.834 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-16T20:22:07.188 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-16T20:22:07.201 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../106-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-16T20:22:07.246 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-16T20:22:07.549 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pytest. 2020-11-16T20:22:07.566 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../107-python3-pytest_3.3.2-2_all.deb ... 2020-11-16T20:22:07.607 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-16T20:22:07.767 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T20:22:07.944 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplejson. 2020-11-16T20:22:07.957 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../108-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-16T20:22:08.002 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-16T20:22:08.346 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-16T20:22:08.362 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../109-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-16T20:22:08.404 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T20:22:08.443 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-16T20:22:08.744 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python-py (1.5.2-1) ... 2020-11-16T20:22:08.867 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package radosgw. 2020-11-16T20:22:08.881 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../110-radosgw_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:22:08.924 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking radosgw (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:09.004 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-16T20:22:09.355 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-16T20:22:09.445 INFO:teuthology.orchestra.run.smithi013.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-16T20:22:09.690 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-16T20:22:09.816 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-16T20:22:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-fuse. 2020-11-16T20:22:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../111-rbd-fuse_16.0.0-7219-g911cc357-1bionic_amd64.deb ... 2020-11-16T20:22:10.009 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-16T20:22:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-fuse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:10.302 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T20:22:10.307 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-16T20:22:10.611 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-16T20:22:10.616 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-16T20:22:10.937 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-16T20:22:11.017 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-16T20:22:11.152 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-16T20:22:11.331 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-16T20:22:11.665 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-16T20:22:11.762 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-16T20:22:11.949 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up cephadm (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:12.155 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-16T20:22:12.456 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-16T20:22:12.652 INFO:teuthology.orchestra.run.smithi013.stdout:Adding system user cephadm....done 2020-11-16T20:22:12.749 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-16T20:22:12.815 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-16T20:22:12.952 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-16T20:22:13.103 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-16T20:22:13.107 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T20:22:13.233 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-16T20:22:13.359 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-16T20:22:13.430 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-16T20:22:13.711 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-16T20:22:13.792 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-16T20:22:14.020 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T20:22:14.074 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-16T20:22:14.196 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-16T20:22:14.375 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-argparse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:14.531 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-ceph-common (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:14.693 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-16T20:22:14.849 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-16T20:22:14.986 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-16T20:22:15.112 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-16T20:22:15.133 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T20:22:15.259 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T20:22:15.627 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-16T20:22:15.995 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T20:22:16.056 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-16T20:22:16.102 INFO:teuthology.orchestra.run.smithi013.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-16T20:22:16.134 INFO:teuthology.orchestra.run.smithi013.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-16T20:22:16.372 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-16T20:22:16.374 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-16T20:22:16.659 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-py (1.5.2-1) ... 2020-11-16T20:22:16.673 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T20:22:16.893 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-16T20:22:16.975 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T20:22:17.228 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-16T20:22:17.317 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-16T20:22:17.334 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-16T20:22:17.580 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-16T20:22:17.660 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-16T20:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-16T20:22:17.886 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T20:22:17.932 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-16T20:22:18.224 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-16T20:22:18.305 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T20:22:18.394 INFO:teuthology.orchestra.run.smithi013.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-16T20:22:18.534 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-16T20:22:18.632 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-16T20:22:18.783 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T20:22:18.818 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-16T20:22:18.892 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T20:22:19.143 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T20:22:19.161 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-16T20:22:19.269 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-16T20:22:19.513 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-16T20:22:19.671 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T20:22:19.797 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-16T20:22:19.813 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up cephadm (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:20.081 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-16T20:22:20.357 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T20:22:20.498 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user cephadm....done 2020-11-16T20:22:20.642 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-16T20:22:20.661 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-16T20:22:20.934 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-16T20:22:20.963 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-16T20:22:21.089 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-16T20:22:21.215 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-16T20:22:21.428 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-16T20:22:21.557 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-16T20:22:21.746 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-16T20:22:21.867 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-16T20:22:22.001 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-16T20:22:22.055 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-16T20:22:22.336 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-common (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:22.390 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T20:22:22.662 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-16T20:22:22.725 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-16T20:22:22.955 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-16T20:22:23.018 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-16T20:22:23.081 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-16T20:22:23.177 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T20:22:23.441 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-16T20:22:23.792 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-16T20:22:23.882 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-16T20:22:23.923 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-16T20:22:24.161 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-16T20:22:24.365 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T20:22:24.470 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-16T20:22:24.675 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-16T20:22:24.771 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-16T20:22:25.035 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-16T20:22:25.122 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-16T20:22:25.161 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-16T20:22:25.423 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-16T20:22:25.529 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-16T20:22:25.624 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-16T20:22:25.780 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T20:22:25.898 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-16T20:22:25.999 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up librados2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:26.043 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-16T20:22:26.100 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-16T20:22:26.166 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-16T20:22:26.401 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up libcephfs2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:26.428 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-16T20:22:26.527 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up libcephfs-dev (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:26.604 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-16T20:22:26.653 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-rados (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:26.730 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-16T20:22:26.804 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T20:22:26.998 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-16T20:22:27.124 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-16T20:22:27.130 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-16T20:22:27.439 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-16T20:22:27.533 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-16T20:22:27.668 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-16T20:22:27.749 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T20:22:27.969 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-16T20:22:28.262 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-16T20:22:28.571 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-16T20:22:28.687 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-16T20:22:28.872 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-16T20:22:29.232 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-16T20:22:29.541 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-16T20:22:29.550 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T20:22:29.818 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-16T20:22:29.826 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up libradosstriper1 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:29.952 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-cephfs (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:30.253 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up librgw2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:30.404 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-fuse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:30.494 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-16T20:22:30.634 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T20:22:30.635 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T20:22:30.871 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-16T20:22:30.914 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up librbd1 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:31.031 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-16T20:22:31.104 INFO:teuthology.orchestra.run.smithi013.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-16T20:22:31.145 INFO:teuthology.orchestra.run.smithi013.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-16T20:22:31.171 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-16T20:22:31.298 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-16T20:22:31.443 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T20:22:32.137 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mgr-modules-core (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:32.271 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-16T20:22:32.461 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-16T20:22:32.589 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-rgw (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:32.699 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up rbd-fuse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:32.779 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-16T20:22:32.799 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up python3-rbd (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:32.901 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-common (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:33.105 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-16T20:22:33.256 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-16T20:22:33.443 INFO:teuthology.orchestra.run.smithi013.stdout:Adding group ceph....done 2020-11-16T20:22:33.632 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-16T20:22:33.867 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-16T20:22:33.905 INFO:teuthology.orchestra.run.smithi013.stdout:Adding system user ceph....done 2020-11-16T20:22:33.976 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-16T20:22:34.085 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librados2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:34.105 INFO:teuthology.orchestra.run.smithi013.stdout:Setting system user ceph properties....done 2020-11-16T20:22:34.212 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-16T20:22:34.266 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-16T20:22:34.488 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-16T20:22:34.521 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:34.663 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs-dev (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:34.789 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rados (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:34.803 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up radosgw (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:34.915 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-16T20:22:35.057 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T20:22:35.057 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T20:22:35.216 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-16T20:22:35.488 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-test (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:35.492 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-16T20:22:35.614 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-base (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:35.827 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-16T20:22:35.984 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-16T20:22:36.257 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mgr (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:36.460 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T20:22:36.461 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T20:22:36.742 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:36.773 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-16T20:22:36.860 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-osd (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:37.313 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T20:22:37.313 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T20:22:37.603 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mgr-cephadm (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:37.635 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-16T20:22:37.738 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mgr-rook (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:37.881 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mds (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:37.953 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libradosstriper1 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:38.079 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cephfs (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:38.102 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T20:22:38.102 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T20:22:38.364 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librgw2 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:38.382 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mon (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:38.489 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-fuse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:38.596 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T20:22:38.597 INFO:teuthology.orchestra.run.smithi013.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T20:22:38.700 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T20:22:38.701 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-16T20:22:38.966 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librbd1 (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:39.067 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-16T20:22:39.166 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-16T20:22:39.198 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-16T20:22:39.358 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:39.469 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph-mgr-dashboard (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:39.504 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-16T20:22:39.611 INFO:teuthology.orchestra.run.smithi013.stdout:Setting up ceph (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:39.738 INFO:teuthology.orchestra.run.smithi013.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T20:22:39.948 INFO:teuthology.orchestra.run.smithi013.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T20:22:40.164 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-modules-core (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:40.315 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-16T20:22:40.617 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rgw (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:40.718 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-fuse (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:40.827 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rbd (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:40.953 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-common (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:41.520 INFO:teuthology.orchestra.run.smithi092.stdout:Adding group ceph....done 2020-11-16T20:22:41.831 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user ceph....done 2020-11-16T20:22:42.032 INFO:teuthology.orchestra.run.smithi092.stdout:Setting system user ceph properties....done 2020-11-16T20:22:42.066 INFO:teuthology.orchestra.run.smithi013.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T20:22:42.100 INFO:teuthology.orchestra.run.smithi013.stdout:ureadahead will be reprofiled on next reboot 2020-11-16T20:22:42.150 INFO:teuthology.orchestra.run.smithi013.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T20:22:42.184 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-16T20:22:42.410 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-16T20:22:42.583 INFO:teuthology.orchestra.run.smithi013.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T20:22:42.586 DEBUG:teuthology.parallel:result is None 2020-11-16T20:22:42.704 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up radosgw (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:42.964 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T20:22:42.965 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-16T20:22:43.389 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-test (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:43.515 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-base (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:43.850 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-16T20:22:44.134 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:44.343 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T20:22:44.343 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-16T20:22:44.643 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:44.786 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-osd (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:45.283 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T20:22:45.284 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-16T20:22:45.581 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-cephadm (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:45.723 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-rook (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:45.866 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mds (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:46.065 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T20:22:46.066 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-16T20:22:46.392 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mon (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:46.593 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T20:22:46.594 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-16T20:22:46.877 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:47.002 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-dashboard (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:47.145 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph (16.0.0-7219-g911cc357-1bionic) ... 2020-11-16T20:22:47.272 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-16T20:22:47.481 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-16T20:22:49.508 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-16T20:22:49.550 INFO:teuthology.orchestra.run.smithi092.stdout:ureadahead will be reprofiled on next reboot 2020-11-16T20:22:49.592 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-16T20:22:50.058 INFO:teuthology.orchestra.run.smithi092.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-16T20:22:50.060 DEBUG:teuthology.parallel:result is None 2020-11-16T20:22:50.061 INFO:teuthology.orchestra.run.smithi092:> sudo lsb_release -is 2020-11-16T20:22:50.106 INFO:teuthology.orchestra.run.smithi092.stdout:Ubuntu 2020-11-16T20:22:50.110 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T20:22:50.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=911cc35794d871071a8bebfb0919482263b1d18b 2020-11-16T20:22:50.279 INFO:teuthology.orchestra.run.smithi092:> dpkg-query -W -f '${Version}' ceph 2020-11-16T20:22:50.302 INFO:teuthology.orchestra.run.smithi092.stdout:16.0.0-7219-g911cc357-1bionic 2020-11-16T20:22:50.303 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7219-g911cc357-1bionic 2020-11-16T20:22:50.303 INFO:teuthology.task.install:The correct ceph version 16.0.0-7219-g911cc357-1bionic is installed. 2020-11-16T20:22:50.303 INFO:teuthology.orchestra.run.smithi013:> sudo lsb_release -is 2020-11-16T20:22:50.351 INFO:teuthology.orchestra.run.smithi013.stdout:Ubuntu 2020-11-16T20:22:50.356 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-16T20:22:50.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=911cc35794d871071a8bebfb0919482263b1d18b 2020-11-16T20:22:50.513 INFO:teuthology.orchestra.run.smithi013:> dpkg-query -W -f '${Version}' ceph 2020-11-16T20:22:50.538 INFO:teuthology.orchestra.run.smithi013.stdout:16.0.0-7219-g911cc357-1bionic 2020-11-16T20:22:50.538 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7219-g911cc357-1bionic 2020-11-16T20:22:50.538 INFO:teuthology.task.install:The correct ceph version 16.0.0-7219-g911cc357-1bionic is installed. 2020-11-16T20:22:50.539 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-16T20:22:50.539 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:50.539 INFO:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T20:22:50.544 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T20:22:50.553 INFO:teuthology.orchestra.run.smithi092.stderr:24+1 records in 2020-11-16T20:22:50.553 INFO:teuthology.orchestra.run.smithi092.stderr:24+1 records out 2020-11-16T20:22:50.553 INFO:teuthology.orchestra.run.smithi092.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000140648 s, 90.1 MB/s 2020-11-16T20:22:50.554 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:22:50.555 INFO:teuthology.orchestra.run.smithi013:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T20:22:50.584 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-16T20:22:50.594 INFO:teuthology.orchestra.run.smithi013.stderr:24+1 records in 2020-11-16T20:22:50.594 INFO:teuthology.orchestra.run.smithi013.stderr:24+1 records out 2020-11-16T20:22:50.594 INFO:teuthology.orchestra.run.smithi013.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.00022103 s, 57.3 MB/s 2020-11-16T20:22:50.595 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-16T20:22:50.595 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:50.596 INFO:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T20:22:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T20:22:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:5+1 records in 2020-11-16T20:22:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:5+1 records out 2020-11-16T20:22:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 9.3929e-05 s, 32.6 MB/s 2020-11-16T20:22:50.615 INFO:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T20:22:50.671 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:22:50.671 INFO:teuthology.orchestra.run.smithi013:> sudo dd of=/usr/bin/daemon-helper 2020-11-16T20:22:50.676 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-16T20:22:50.685 INFO:teuthology.orchestra.run.smithi013.stderr:5+1 records in 2020-11-16T20:22:50.685 INFO:teuthology.orchestra.run.smithi013.stderr:5+1 records out 2020-11-16T20:22:50.686 INFO:teuthology.orchestra.run.smithi013.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 0.00010203 s, 30.0 MB/s 2020-11-16T20:22:50.686 INFO:teuthology.orchestra.run.smithi013:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-16T20:22:50.742 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-16T20:22:50.742 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:50.743 INFO:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T20:22:50.747 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T20:22:50.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:22:50.757 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:22:50.757 INFO:teuthology.orchestra.run.smithi092.stderr:299 bytes copied, 6.892e-05 s, 4.3 MB/s 2020-11-16T20:22:50.758 INFO:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T20:22:50.815 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:22:50.815 INFO:teuthology.orchestra.run.smithi013:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T20:22:50.820 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-16T20:22:50.829 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:22:50.829 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:22:50.829 INFO:teuthology.orchestra.run.smithi013.stderr:299 bytes copied, 7.5497e-05 s, 4.0 MB/s 2020-11-16T20:22:50.830 INFO:teuthology.orchestra.run.smithi013:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-16T20:22:50.886 INFO:teuthology.run_tasks:Running task ceph... 2020-11-16T20:22:50.942 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-16T20:22:50.943 INFO:teuthology.orchestra.run.smithi013:> sudo chmod 777 /var/log/ceph 2020-11-16T20:22:50.945 INFO:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2020-11-16T20:22:50.959 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-16T20:22:50.960 INFO:teuthology.orchestra.run.smithi013:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T20:22:51.002 INFO:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-16T20:22:51.016 INFO:tasks.ceph:Creating extra log directories... 2020-11-16T20:22:51.017 INFO:teuthology.orchestra.run.smithi013:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T20:22:51.057 INFO:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-16T20:22:51.113 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-16T20:22:51.113 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'Health detail', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-16T20:22:51.113 INFO:tasks.ceph:ctx.config {'arch': 'x86_64', 'archive_path': '/home/teuthworker/archive/kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi/5631141', 'branch': 'wip-kefu-testing-2020-11-16-1845', 'description': 'rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}', 'email': 'tchaikov@gmail.com', 'first_in_suite': False, 'job_id': '5631141', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'kchai-2020-11-16_12:11:03-rados-wip-kefu-testing-2020-11-16-1845-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-kefu-testing-2020-11-16-1845'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'Health detail', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'Health detail'], 'sha1': '911cc35794d871071a8bebfb0919482263b1d18b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '911cc35794d871071a8bebfb0919482263b1d18b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-kefu-testing-2020-11-16-1845', 'sha1': '911cc35794d871071a8bebfb0919482263b1d18b'}}, 'owner': 'scheduled_kchai@teuthology', 'priority': 90, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '911cc35794d871071a8bebfb0919482263b1d18b', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-kefu-testing-2020-11-16-1845', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-11-16-1845/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '911cc35794d871071a8bebfb0919482263b1d18b', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'Health detail', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'Health detail'], 'sha1': '911cc35794d871071a8bebfb0919482263b1d18b', 'cluster': 'ceph'}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 2, 'm': 1, 'name': 'isaprofile', 'plugin': 'isa', 'technique': 'reed_sol_van'}, 'min_size': 2, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 4000, 'write_append_excl': False}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-16_12:11:03', 'tube': 'smithi', 'user': 'kchai', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.7053', 'targets': {'smithi092.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKWHue7qo4kG7vXGAjFr6xMwF4Q1TyjEPt99U/vtRzjOsmbiEb4jSr7bxB1LcmqKR3oXnLUN69JwsszKPNfgI8BXSofGk3OrQisrBcaXFbeuzulKMSGGfFQYrNFiacFETOkbL6D9r9qhyNQZegacGZPCy67Wb3u82T8IqpiWcN+v3UkOB+t0SOGkdAMs1rUxmZmuGc+bFSS9c1TRyRrohoWfT9xi9VrUN2u7CJ7h9M+yDxIHhMxM9zxgo3vct6qY/htTNo8PQ3CG/esEykxy0lS8Sud2+we2fXph/weWmJhIw3VrWZrxBrwbdxOk8owIf657pE5q5J4xpztJLg9Bax', 'smithi013.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCv1BfmhdKqKp0UJPrCQbmfkTwciMfmtzxUKy2fa+G04S2O/6QQAgY5Du51Q5EnMe1FWmE+2lwJiTCVIt8tVaFCjiMcHDOSWap5N6BtHMR2ylsQrKVy+8esDehTiiT9BKHRY+5rk/wadBy+NhTVFgUQKZQZOt/iHQhNtr/89CTDBttDO3VOpIQOYqhOYBdPM7FO/FQ2VxhX6TT+sO6dJZ9VdVIpSsXHyzVK7wvRgzJ7JuOIy68wM8GBu9PLMZ9+GgvImX6SAw0PPT2fACDBymSBIEGogJXqvdsU8sRbhT2OQJQOLaT7doWCjMMIBmegEDraQ8VrRxJsjh0wrIn59823'}} 2020-11-16T20:22:51.114 INFO:teuthology.orchestra.run.smithi013:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T20:22:51.116 INFO:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-16T20:22:51.123 INFO:teuthology.orchestra.run.smithi013:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T20:22:51.165 INFO:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2020-11-16T20:22:51.180 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:51.181 INFO:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T20:22:51.225 INFO:teuthology.orchestra.run.smithi092.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T20:22:51.226 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/vg_nvme/lv_1 2020-11-16T20:22:51.226 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/vg_nvme/lv_2 2020-11-16T20:22:51.226 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/vg_nvme/lv_3 2020-11-16T20:22:51.226 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/vg_nvme/lv_4 2020-11-16T20:22:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:22:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:22:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:72 bytes copied, 0.000136467 s, 528 kB/s 2020-11-16T20:22:51.227 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T20:22:51.227 INFO:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T20:22:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T20:22:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T20:22:51.276 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-11-16T20:22:51.276 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T20:22:51.276 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2020-11-16 20:19:49.071732080 +0000 2020-11-16T20:22:51.276 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2020-11-16 20:19:48.955735642 +0000 2020-11-16T20:22:51.277 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2020-11-16 20:19:48.955735642 +0000 2020-11-16T20:22:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2020-11-16T20:22:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2020-11-16T20:22:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2020-11-16T20:22:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000411623 s, 1.2 MB/s 2020-11-16T20:22:51.290 INFO:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T20:22:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T20:22:51.340 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T20:22:51.340 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 348 Links: 1 2020-11-16T20:22:51.340 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T20:22:51.340 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2020-11-16 20:19:49.323724341 +0000 2020-11-16T20:22:51.341 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2020-11-16 20:19:49.207727902 +0000 2020-11-16T20:22:51.341 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2020-11-16 20:19:49.207727902 +0000 2020-11-16T20:22:51.341 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2020-11-16T20:22:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2020-11-16T20:22:51.349 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2020-11-16T20:22:51.350 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000374557 s, 1.4 MB/s 2020-11-16T20:22:51.353 INFO:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T20:22:51.403 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T20:22:51.404 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T20:22:51.404 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 354 Links: 1 2020-11-16T20:22:51.404 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T20:22:51.404 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2020-11-16 20:19:49.547717461 +0000 2020-11-16T20:22:51.404 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2020-11-16 20:19:49.439720777 +0000 2020-11-16T20:22:51.405 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2020-11-16 20:19:49.439720777 +0000 2020-11-16T20:22:51.405 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2020-11-16T20:22:51.413 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2020-11-16T20:22:51.413 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2020-11-16T20:22:51.413 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000358306 s, 1.4 MB/s 2020-11-16T20:22:51.418 INFO:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T20:22:51.467 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T20:22:51.468 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T20:22:51.468 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-11-16T20:22:51.468 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T20:22:51.468 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2020-11-16 20:19:49.795709844 +0000 2020-11-16T20:22:51.468 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2020-11-16 20:19:49.683713284 +0000 2020-11-16T20:22:51.468 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2020-11-16 20:19:49.683713284 +0000 2020-11-16T20:22:51.469 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2020-11-16T20:22:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2020-11-16T20:22:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2020-11-16T20:22:51.477 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000399344 s, 1.3 MB/s 2020-11-16T20:22:51.481 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T20:22:51.482 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:22:51.482 INFO:teuthology.orchestra.run.smithi013:> dd if=/scratch_devs of=/dev/stdout 2020-11-16T20:22:51.487 INFO:teuthology.orchestra.run.smithi013.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-16T20:22:51.492 INFO:teuthology.orchestra.run.smithi013.stdout:/dev/vg_nvme/lv_1 2020-11-16T20:22:51.493 INFO:teuthology.orchestra.run.smithi013.stdout:/dev/vg_nvme/lv_2 2020-11-16T20:22:51.493 INFO:teuthology.orchestra.run.smithi013.stdout:/dev/vg_nvme/lv_3 2020-11-16T20:22:51.493 INFO:teuthology.orchestra.run.smithi013.stdout:/dev/vg_nvme/lv_4 2020-11-16T20:22:51.493 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:22:51.493 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:22:51.493 INFO:teuthology.orchestra.run.smithi013.stderr:72 bytes copied, 0.000176007 s, 409 kB/s 2020-11-16T20:22:51.494 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-16T20:22:51.494 INFO:teuthology.orchestra.run.smithi013:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-16T20:22:51.542 INFO:teuthology.orchestra.run.smithi013.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-16T20:22:51.543 INFO:teuthology.orchestra.run.smithi013.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T20:22:51.543 INFO:teuthology.orchestra.run.smithi013.stdout:Device: 6h/6d Inode: 341 Links: 1 2020-11-16T20:22:51.543 INFO:teuthology.orchestra.run.smithi013.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T20:22:51.543 INFO:teuthology.orchestra.run.smithi013.stdout:Access: 2020-11-16 20:19:55.576231827 +0000 2020-11-16T20:22:51.543 INFO:teuthology.orchestra.run.smithi013.stdout:Modify: 2020-11-16 20:19:55.456232753 +0000 2020-11-16T20:22:51.543 INFO:teuthology.orchestra.run.smithi013.stdout:Change: 2020-11-16 20:19:55.456232753 +0000 2020-11-16T20:22:51.544 INFO:teuthology.orchestra.run.smithi013.stdout: Birth: - 2020-11-16T20:22:51.552 INFO:teuthology.orchestra.run.smithi013.stderr:1+0 records in 2020-11-16T20:22:51.552 INFO:teuthology.orchestra.run.smithi013.stderr:1+0 records out 2020-11-16T20:22:51.553 INFO:teuthology.orchestra.run.smithi013.stderr:512 bytes copied, 0.000321083 s, 1.6 MB/s 2020-11-16T20:22:51.557 INFO:teuthology.orchestra.run.smithi013:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-16T20:22:51.606 INFO:teuthology.orchestra.run.smithi013.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-16T20:22:51.607 INFO:teuthology.orchestra.run.smithi013.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T20:22:51.607 INFO:teuthology.orchestra.run.smithi013.stdout:Device: 6h/6d Inode: 347 Links: 1 2020-11-16T20:22:51.607 INFO:teuthology.orchestra.run.smithi013.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T20:22:51.607 INFO:teuthology.orchestra.run.smithi013.stdout:Access: 2020-11-16 20:19:55.808230036 +0000 2020-11-16T20:22:51.607 INFO:teuthology.orchestra.run.smithi013.stdout:Modify: 2020-11-16 20:19:55.708230808 +0000 2020-11-16T20:22:51.608 INFO:teuthology.orchestra.run.smithi013.stdout:Change: 2020-11-16 20:19:55.708230808 +0000 2020-11-16T20:22:51.608 INFO:teuthology.orchestra.run.smithi013.stdout: Birth: - 2020-11-16T20:22:51.615 INFO:teuthology.orchestra.run.smithi013.stderr:1+0 records in 2020-11-16T20:22:51.615 INFO:teuthology.orchestra.run.smithi013.stderr:1+0 records out 2020-11-16T20:22:51.616 INFO:teuthology.orchestra.run.smithi013.stderr:512 bytes copied, 0.000267101 s, 1.9 MB/s 2020-11-16T20:22:51.620 INFO:teuthology.orchestra.run.smithi013:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-16T20:22:51.666 INFO:teuthology.orchestra.run.smithi013.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-16T20:22:51.667 INFO:teuthology.orchestra.run.smithi013.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T20:22:51.667 INFO:teuthology.orchestra.run.smithi013.stdout:Device: 6h/6d Inode: 354 Links: 1 2020-11-16T20:22:51.667 INFO:teuthology.orchestra.run.smithi013.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T20:22:51.667 INFO:teuthology.orchestra.run.smithi013.stdout:Access: 2020-11-16 20:19:56.028228338 +0000 2020-11-16T20:22:51.667 INFO:teuthology.orchestra.run.smithi013.stdout:Modify: 2020-11-16 20:19:55.948228956 +0000 2020-11-16T20:22:51.667 INFO:teuthology.orchestra.run.smithi013.stdout:Change: 2020-11-16 20:19:55.948228956 +0000 2020-11-16T20:22:51.668 INFO:teuthology.orchestra.run.smithi013.stdout: Birth: - 2020-11-16T20:22:51.675 INFO:teuthology.orchestra.run.smithi013.stderr:1+0 records in 2020-11-16T20:22:51.675 INFO:teuthology.orchestra.run.smithi013.stderr:1+0 records out 2020-11-16T20:22:51.676 INFO:teuthology.orchestra.run.smithi013.stderr:512 bytes copied, 0.000258981 s, 2.0 MB/s 2020-11-16T20:22:51.680 INFO:teuthology.orchestra.run.smithi013:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-16T20:22:51.726 INFO:teuthology.orchestra.run.smithi013.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-16T20:22:51.727 INFO:teuthology.orchestra.run.smithi013.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-16T20:22:51.727 INFO:teuthology.orchestra.run.smithi013.stdout:Device: 6h/6d Inode: 360 Links: 1 2020-11-16T20:22:51.727 INFO:teuthology.orchestra.run.smithi013.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-16T20:22:51.727 INFO:teuthology.orchestra.run.smithi013.stdout:Access: 2020-11-16 20:19:56.268226486 +0000 2020-11-16T20:22:51.727 INFO:teuthology.orchestra.run.smithi013.stdout:Modify: 2020-11-16 20:19:56.156227351 +0000 2020-11-16T20:22:51.727 INFO:teuthology.orchestra.run.smithi013.stdout:Change: 2020-11-16 20:19:56.156227351 +0000 2020-11-16T20:22:51.728 INFO:teuthology.orchestra.run.smithi013.stdout: Birth: - 2020-11-16T20:22:51.736 INFO:teuthology.orchestra.run.smithi013.stderr:1+0 records in 2020-11-16T20:22:51.736 INFO:teuthology.orchestra.run.smithi013.stderr:1+0 records out 2020-11-16T20:22:51.736 INFO:teuthology.orchestra.run.smithi013.stderr:512 bytes copied, 0.00028493 s, 1.8 MB/s 2020-11-16T20:22:51.741 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T20:22:51.741 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi013.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-16T20:22:51.741 INFO:tasks.ceph:Generating config... 2020-11-16T20:22:51.743 INFO:tasks.ceph:[osd] debug monc = 20 2020-11-16T20:22:51.743 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-16T20:22:51.743 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2020-11-16T20:22:51.743 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2020-11-16T20:22:51.743 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-16T20:22:51.744 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-16T20:22:51.744 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-16T20:22:51.744 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-16T20:22:51.744 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-16T20:22:51.744 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-16T20:22:51.745 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-16T20:22:51.745 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-16T20:22:51.745 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-16T20:22:51.745 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-16T20:22:51.745 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-16T20:22:51.746 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2020-11-16T20:22:51.746 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2020-11-16T20:22:51.746 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2020-11-16T20:22:51.746 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-16T20:22:51.747 INFO:tasks.ceph:[osd] osd max backfills = 3 2020-11-16T20:22:51.747 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2020-11-16T20:22:51.747 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-16T20:22:51.747 INFO:tasks.ceph:[osd] osd op queue = debug_random 2020-11-16T20:22:51.747 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2020-11-16T20:22:51.747 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2020-11-16T20:22:51.748 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2020-11-16T20:22:51.748 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2020-11-16T20:22:51.748 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2020-11-16T20:22:51.748 INFO:tasks.ceph:[global] mon client directed command retry = 5 2020-11-16T20:22:51.749 INFO:tasks.ceph:[global] mon election default strategy = 1 2020-11-16T20:22:51.749 INFO:tasks.ceph:[global] ms inject delay max = 1 2020-11-16T20:22:51.749 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2020-11-16T20:22:51.749 INFO:tasks.ceph:[global] ms inject delay type = osd 2020-11-16T20:22:51.749 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2020-11-16T20:22:51.750 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2020-11-16T20:22:51.750 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2020-11-16T20:22:51.750 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-16T20:22:51.750 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-16T20:22:51.750 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-16T20:22:51.751 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-16T20:22:51.751 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-16T20:22:51.751 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2020-11-16T20:22:51.751 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2020-11-16T20:22:51.751 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2020-11-16T20:22:51.751 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2020-11-16T20:22:51.752 INFO:tasks.ceph:[mon] paxos service trim min = 10 2020-11-16T20:22:51.752 INFO:tasks.ceph:Setting up mon.a... 2020-11-16T20:22:51.752 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-16T20:22:51.785 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2020-11-16T20:22:51.787 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-16T20:22:51.862 INFO:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T20:22:51.917 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.c', '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]'), ('mon.b', '172.21.15.13')] 2020-11-16T20:22:51.918 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.13', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2020-11-16T20:22:51.919 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:51.919 INFO:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T20:22:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:+ dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T20:22:51.967 INFO:teuthology.orchestra.run.smithi092.stderr:7+1 records in 2020-11-16T20:22:51.967 INFO:teuthology.orchestra.run.smithi092.stderr:7+1 records out 2020-11-16T20:22:51.967 INFO:teuthology.orchestra.run.smithi092.stderr:3719 bytes (3.7 kB, 3.6 KiB) copied, 0.00011794 s, 31.5 MB/s 2020-11-16T20:22:51.967 INFO:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --addv c '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]' --add b 172.21.15.13 --print /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:22:52.045 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:22:52.045 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 3fc8a474-4fd1-4d15-a656-0a82bbfce204 2020-11-16T20:22:52.046 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2020-11-16T20:22:52.046 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 3fc8a474-4fd1-4d15-a656-0a82bbfce204 2020-11-16T20:22:52.046 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2020-11-16T20:22:52.051022+0000 2020-11-16T20:22:52.046 INFO:teuthology.orchestra.run.smithi092.stdout:created 2020-11-16T20:22:52.051022+0000 2020-11-16T20:22:52.046 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 0 (unknown) 2020-11-16T20:22:52.046 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2020-11-16T20:22:52.047 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2020-11-16T20:22:52.047 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.13:3300/0,v1:172.21.15.13:6789/0] mon.b 2020-11-16T20:22:52.047 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] mon.c 2020-11-16T20:22:52.047 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-16T20:22:52.048 INFO:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2020-11-16T20:22:52.095 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3fc8a474-4fd1-4d15-a656-0a82bbfce204... 2020-11-16T20:22:52.096 INFO:teuthology.orchestra.run.smithi013:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T20:22:52.098 INFO:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-16T20:22:52.139 INFO:teuthology.orchestra.run.smithi013.stdout:[global] 2020-11-16T20:22:52.140 INFO:teuthology.orchestra.run.smithi013.stdout: chdir = "" 2020-11-16T20:22:52.140 INFO:teuthology.orchestra.run.smithi013.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T20:22:52.140 INFO:teuthology.orchestra.run.smithi013.stdout: auth supported = cephx 2020-11-16T20:22:52.140 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.140 INFO:teuthology.orchestra.run.smithi013.stdout: filestore xattr use omap = true 2020-11-16T20:22:52.140 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.141 INFO:teuthology.orchestra.run.smithi013.stdout: mon clock drift allowed = 1.000 2020-11-16T20:22:52.141 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.141 INFO:teuthology.orchestra.run.smithi013.stdout: osd crush chooseleaf type = 0 2020-11-16T20:22:52.141 INFO:teuthology.orchestra.run.smithi013.stdout: auth debug = true 2020-11-16T20:22:52.141 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.141 INFO:teuthology.orchestra.run.smithi013.stdout: ms die on old message = true 2020-11-16T20:22:52.141 INFO:teuthology.orchestra.run.smithi013.stdout: ms die on bug = true 2020-11-16T20:22:52.142 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.142 INFO:teuthology.orchestra.run.smithi013.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T20:22:52.142 INFO:teuthology.orchestra.run.smithi013.stdout: mon pg warn max object skew = 0 2020-11-16T20:22:52.142 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.142 INFO:teuthology.orchestra.run.smithi013.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T20:22:52.142 INFO:teuthology.orchestra.run.smithi013.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T20:22:52.142 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.143 INFO:teuthology.orchestra.run.smithi013.stdout: osd pool default size = 2 2020-11-16T20:22:52.143 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.143 INFO:teuthology.orchestra.run.smithi013.stdout: mon osd allow primary affinity = true 2020-11-16T20:22:52.143 INFO:teuthology.orchestra.run.smithi013.stdout: mon osd allow pg remap = true 2020-11-16T20:22:52.143 INFO:teuthology.orchestra.run.smithi013.stdout: mon warn on legacy crush tunables = false 2020-11-16T20:22:52.143 INFO:teuthology.orchestra.run.smithi013.stdout: mon warn on crush straw calc version zero = false 2020-11-16T20:22:52.143 INFO:teuthology.orchestra.run.smithi013.stdout: mon warn on no sortbitwise = false 2020-11-16T20:22:52.143 INFO:teuthology.orchestra.run.smithi013.stdout: mon warn on osd down out interval zero = false 2020-11-16T20:22:52.144 INFO:teuthology.orchestra.run.smithi013.stdout: mon warn on too few osds = false 2020-11-16T20:22:52.144 INFO:teuthology.orchestra.run.smithi013.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T20:22:52.144 INFO:teuthology.orchestra.run.smithi013.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T20:22:52.144 INFO:teuthology.orchestra.run.smithi013.stdout: mon_allow_pool_size_one = true 2020-11-16T20:22:52.144 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.144 INFO:teuthology.orchestra.run.smithi013.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T20:22:52.144 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.145 INFO:teuthology.orchestra.run.smithi013.stdout: osd default data pool replay window = 5 2020-11-16T20:22:52.145 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.145 INFO:teuthology.orchestra.run.smithi013.stdout: mon allow pool delete = true 2020-11-16T20:22:52.145 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.145 INFO:teuthology.orchestra.run.smithi013.stdout: mon cluster log file level = debug 2020-11-16T20:22:52.145 INFO:teuthology.orchestra.run.smithi013.stdout: debug asserts on shutdown = true 2020-11-16T20:22:52.145 INFO:teuthology.orchestra.run.smithi013.stdout: mon health detail to clog = false 2020-11-16T20:22:52.145 INFO:teuthology.orchestra.run.smithi013.stdout: mon host = "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.13" 2020-11-16T20:22:52.146 INFO:teuthology.orchestra.run.smithi013.stdout: mon client directed command retry = 5 2020-11-16T20:22:52.146 INFO:teuthology.orchestra.run.smithi013.stdout: mon election default strategy = 1 2020-11-16T20:22:52.146 INFO:teuthology.orchestra.run.smithi013.stdout: ms inject delay max = 1 2020-11-16T20:22:52.146 INFO:teuthology.orchestra.run.smithi013.stdout: ms inject delay probability = 0.005 2020-11-16T20:22:52.146 INFO:teuthology.orchestra.run.smithi013.stdout: ms inject delay type = osd 2020-11-16T20:22:52.146 INFO:teuthology.orchestra.run.smithi013.stdout: ms inject internal delays = 0.002 2020-11-16T20:22:52.146 INFO:teuthology.orchestra.run.smithi013.stdout: ms inject socket failures = 2500 2020-11-16T20:22:52.147 INFO:teuthology.orchestra.run.smithi013.stdout: osd_async_recovery_min_cost = 1 2020-11-16T20:22:52.147 INFO:teuthology.orchestra.run.smithi013.stdout: fsid = 3fc8a474-4fd1-4d15-a656-0a82bbfce204 2020-11-16T20:22:52.147 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.147 INFO:teuthology.orchestra.run.smithi013.stdout:[osd] 2020-11-16T20:22:52.147 INFO:teuthology.orchestra.run.smithi013.stdout: osd journal size = 100 2020-11-16T20:22:52.147 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.147 INFO:teuthology.orchestra.run.smithi013.stdout: osd scrub load threshold = 5.0 2020-11-16T20:22:52.148 INFO:teuthology.orchestra.run.smithi013.stdout: osd scrub max interval = 120 2020-11-16T20:22:52.148 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.148 INFO:teuthology.orchestra.run.smithi013.stdout: osd recover clone overlap = true 2020-11-16T20:22:52.148 INFO:teuthology.orchestra.run.smithi013.stdout: osd recovery max chunk = 1048576 2020-11-16T20:22:52.148 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.148 INFO:teuthology.orchestra.run.smithi013.stdout: osd debug shutdown = true 2020-11-16T20:22:52.148 INFO:teuthology.orchestra.run.smithi013.stdout: osd debug op order = true 2020-11-16T20:22:52.148 INFO:teuthology.orchestra.run.smithi013.stdout: osd debug verify stray on activate = true 2020-11-16T20:22:52.149 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.149 INFO:teuthology.orchestra.run.smithi013.stdout: osd open classes on start = true 2020-11-16T20:22:52.149 INFO:teuthology.orchestra.run.smithi013.stdout: osd debug pg log writeout = true 2020-11-16T20:22:52.149 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.149 INFO:teuthology.orchestra.run.smithi013.stdout: osd deep scrub update digest min age = 30 2020-11-16T20:22:52.149 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.149 INFO:teuthology.orchestra.run.smithi013.stdout: osd map max advance = 10 2020-11-16T20:22:52.150 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.150 INFO:teuthology.orchestra.run.smithi013.stdout: journal zero on create = true 2020-11-16T20:22:52.150 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.150 INFO:teuthology.orchestra.run.smithi013.stdout: filestore ondisk finisher threads = 3 2020-11-16T20:22:52.150 INFO:teuthology.orchestra.run.smithi013.stdout: filestore apply finisher threads = 3 2020-11-16T20:22:52.150 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.150 INFO:teuthology.orchestra.run.smithi013.stdout: bdev debug aio = true 2020-11-16T20:22:52.150 INFO:teuthology.orchestra.run.smithi013.stdout: osd debug misdirected ops = true 2020-11-16T20:22:52.151 INFO:teuthology.orchestra.run.smithi013.stdout: debug monc = 20 2020-11-16T20:22:52.151 INFO:teuthology.orchestra.run.smithi013.stdout: bluestore block size = 96636764160 2020-11-16T20:22:52.151 INFO:teuthology.orchestra.run.smithi013.stdout: bluestore compression algorithm = zlib 2020-11-16T20:22:52.151 INFO:teuthology.orchestra.run.smithi013.stdout: bluestore compression mode = aggressive 2020-11-16T20:22:52.151 INFO:teuthology.orchestra.run.smithi013.stdout: bluestore fsck on mount = True 2020-11-16T20:22:52.151 INFO:teuthology.orchestra.run.smithi013.stdout: debug bluefs = 20 2020-11-16T20:22:52.151 INFO:teuthology.orchestra.run.smithi013.stdout: debug bluestore = 20 2020-11-16T20:22:52.152 INFO:teuthology.orchestra.run.smithi013.stdout: debug filestore = 20 2020-11-16T20:22:52.152 INFO:teuthology.orchestra.run.smithi013.stdout: debug journal = 20 2020-11-16T20:22:52.152 INFO:teuthology.orchestra.run.smithi013.stdout: debug ms = 20 2020-11-16T20:22:52.152 INFO:teuthology.orchestra.run.smithi013.stdout: debug osd = 25 2020-11-16T20:22:52.152 INFO:teuthology.orchestra.run.smithi013.stdout: debug rocksdb = 10 2020-11-16T20:22:52.152 INFO:teuthology.orchestra.run.smithi013.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T20:22:52.152 INFO:teuthology.orchestra.run.smithi013.stdout: mon osd full ratio = 0.9 2020-11-16T20:22:52.153 INFO:teuthology.orchestra.run.smithi013.stdout: mon osd nearfull ratio = 0.8 2020-11-16T20:22:52.153 INFO:teuthology.orchestra.run.smithi013.stdout: osd debug reject backfill probability = 0.3 2020-11-16T20:22:52.153 INFO:teuthology.orchestra.run.smithi013.stdout: osd debug verify cached snaps = True 2020-11-16T20:22:52.153 INFO:teuthology.orchestra.run.smithi013.stdout: osd debug verify missing on start = True 2020-11-16T20:22:52.153 INFO:teuthology.orchestra.run.smithi013.stdout: osd failsafe full ratio = 0.95 2020-11-16T20:22:52.153 INFO:teuthology.orchestra.run.smithi013.stdout: osd max backfills = 3 2020-11-16T20:22:52.153 INFO:teuthology.orchestra.run.smithi013.stdout: osd max markdown count = 1000 2020-11-16T20:22:52.153 INFO:teuthology.orchestra.run.smithi013.stdout: osd objectstore = bluestore 2020-11-16T20:22:52.154 INFO:teuthology.orchestra.run.smithi013.stdout: osd op queue = debug_random 2020-11-16T20:22:52.154 INFO:teuthology.orchestra.run.smithi013.stdout: osd op queue cut off = debug_random 2020-11-16T20:22:52.154 INFO:teuthology.orchestra.run.smithi013.stdout: osd scrub min interval = 60 2020-11-16T20:22:52.154 INFO:teuthology.orchestra.run.smithi013.stdout: osd shutdown pgref assert = True 2020-11-16T20:22:52.154 INFO:teuthology.orchestra.run.smithi013.stdout: osd snap trim sleep = 2 2020-11-16T20:22:52.154 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.154 INFO:teuthology.orchestra.run.smithi013.stdout:[mgr] 2020-11-16T20:22:52.155 INFO:teuthology.orchestra.run.smithi013.stdout: debug ms = 1 2020-11-16T20:22:52.155 INFO:teuthology.orchestra.run.smithi013.stdout: debug mgr = 20 2020-11-16T20:22:52.155 INFO:teuthology.orchestra.run.smithi013.stdout: debug mon = 20 2020-11-16T20:22:52.155 INFO:teuthology.orchestra.run.smithi013.stdout: debug auth = 20 2020-11-16T20:22:52.155 INFO:teuthology.orchestra.run.smithi013.stdout: mon reweight min pgs per osd = 4 2020-11-16T20:22:52.155 INFO:teuthology.orchestra.run.smithi013.stdout: mon reweight min bytes per osd = 10 2020-11-16T20:22:52.155 INFO:teuthology.orchestra.run.smithi013.stdout: mgr/telemetry/nag = false 2020-11-16T20:22:52.155 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.156 INFO:teuthology.orchestra.run.smithi013.stdout:[mon] 2020-11-16T20:22:52.156 INFO:teuthology.orchestra.run.smithi013.stdout: debug ms = 1 2020-11-16T20:22:52.156 INFO:teuthology.orchestra.run.smithi013.stdout: debug mon = 20 2020-11-16T20:22:52.156 INFO:teuthology.orchestra.run.smithi013.stdout: debug paxos = 20 2020-11-16T20:22:52.156 INFO:teuthology.orchestra.run.smithi013.stdout: debug auth = 20 2020-11-16T20:22:52.156 INFO:teuthology.orchestra.run.smithi013.stdout: mon data avail warn = 5 2020-11-16T20:22:52.156 INFO:teuthology.orchestra.run.smithi013.stdout: mon mgr mkfs grace = 240 2020-11-16T20:22:52.157 INFO:teuthology.orchestra.run.smithi013.stdout: mon reweight min pgs per osd = 4 2020-11-16T20:22:52.157 INFO:teuthology.orchestra.run.smithi013.stdout: mon osd reporter subtree level = osd 2020-11-16T20:22:52.157 INFO:teuthology.orchestra.run.smithi013.stdout: mon osd prime pg temp = true 2020-11-16T20:22:52.157 INFO:teuthology.orchestra.run.smithi013.stdout: mon reweight min bytes per osd = 10 2020-11-16T20:22:52.157 INFO:teuthology.orchestra.run.smithi013.stdout: mon min osdmap epochs = 50 2020-11-16T20:22:52.157 INFO:teuthology.orchestra.run.smithi013.stdout: mon osdmap full prune interval = 2 2020-11-16T20:22:52.157 INFO:teuthology.orchestra.run.smithi013.stdout: mon osdmap full prune min = 15 2020-11-16T20:22:52.157 INFO:teuthology.orchestra.run.smithi013.stdout: mon osdmap full prune txsize = 2 2020-11-16T20:22:52.158 INFO:teuthology.orchestra.run.smithi013.stdout: paxos service trim min = 10 2020-11-16T20:22:52.158 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-11-16T20:22:52.158 INFO:teuthology.orchestra.run.smithi013.stdout:[client] 2020-11-16T20:22:52.158 INFO:teuthology.orchestra.run.smithi013.stdout: rgw cache enabled = true 2020-11-16T20:22:52.158 INFO:teuthology.orchestra.run.smithi013.stdout: rgw enable ops log = true 2020-11-16T20:22:52.158 INFO:teuthology.orchestra.run.smithi013.stdout: rgw enable usage log = true 2020-11-16T20:22:52.158 INFO:teuthology.orchestra.run.smithi013.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T20:22:52.159 INFO:teuthology.orchestra.run.smithi013.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T20:22:52.159 INFO:teuthology.orchestra.run.smithi013.stdout:[mon.a] 2020-11-16T20:22:52.159 INFO:teuthology.orchestra.run.smithi013.stdout:[mon.c] 2020-11-16T20:22:52.159 INFO:teuthology.orchestra.run.smithi013.stdout:[mon.b] 2020-11-16T20:22:52.169 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2020-11-16T20:22:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2020-11-16T20:22:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-16T20:22:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2020-11-16T20:22:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2020-11-16T20:22:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2020-11-16T20:22:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2020-11-16T20:22:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2020-11-16T20:22:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2020-11-16T20:22:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2020-11-16T20:22:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-16T20:22:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2020-11-16T20:22:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2020-11-16T20:22:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-16T20:22:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2020-11-16T20:22:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2020-11-16T20:22:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2020-11-16T20:22:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2020-11-16T20:22:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2020-11-16T20:22:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2020-11-16T20:22:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2020-11-16T20:22:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2020-11-16T20:22:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-16T20:22:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-16T20:22:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2020-11-16T20:22:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-16T20:22:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2020-11-16T20:22:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2020-11-16T20:22:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2020-11-16T20:22:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2020-11-16T20:22:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2020-11-16T20:22:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.13" 2020-11-16T20:22:52.176 INFO:teuthology.orchestra.run.smithi092.stdout: mon client directed command retry = 5 2020-11-16T20:22:52.176 INFO:teuthology.orchestra.run.smithi092.stdout: mon election default strategy = 1 2020-11-16T20:22:52.176 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject delay max = 1 2020-11-16T20:22:52.176 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject delay probability = 0.005 2020-11-16T20:22:52.176 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject delay type = osd 2020-11-16T20:22:52.176 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject internal delays = 0.002 2020-11-16T20:22:52.176 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject socket failures = 2500 2020-11-16T20:22:52.177 INFO:teuthology.orchestra.run.smithi092.stdout: osd_async_recovery_min_cost = 1 2020-11-16T20:22:52.177 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 3fc8a474-4fd1-4d15-a656-0a82bbfce204 2020-11-16T20:22:52.177 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.177 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2020-11-16T20:22:52.177 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2020-11-16T20:22:52.177 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.177 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2020-11-16T20:22:52.177 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 120 2020-11-16T20:22:52.178 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.178 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2020-11-16T20:22:52.178 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2020-11-16T20:22:52.178 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.178 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2020-11-16T20:22:52.178 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2020-11-16T20:22:52.178 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2020-11-16T20:22:52.179 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.179 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2020-11-16T20:22:52.179 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2020-11-16T20:22:52.179 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.179 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2020-11-16T20:22:52.179 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.179 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2020-11-16T20:22:52.179 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.180 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2020-11-16T20:22:52.180 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.180 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2020-11-16T20:22:52.180 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2020-11-16T20:22:52.180 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.180 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2020-11-16T20:22:52.180 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2020-11-16T20:22:52.181 INFO:teuthology.orchestra.run.smithi092.stdout: debug monc = 20 2020-11-16T20:22:52.181 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2020-11-16T20:22:52.181 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore compression algorithm = zlib 2020-11-16T20:22:52.181 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore compression mode = aggressive 2020-11-16T20:22:52.181 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2020-11-16T20:22:52.181 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 20 2020-11-16T20:22:52.181 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 20 2020-11-16T20:22:52.181 INFO:teuthology.orchestra.run.smithi092.stdout: debug filestore = 20 2020-11-16T20:22:52.182 INFO:teuthology.orchestra.run.smithi092.stdout: debug journal = 20 2020-11-16T20:22:52.182 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 20 2020-11-16T20:22:52.182 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 25 2020-11-16T20:22:52.182 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 10 2020-11-16T20:22:52.182 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2020-11-16T20:22:52.182 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2020-11-16T20:22:52.182 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2020-11-16T20:22:52.183 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug reject backfill probability = 0.3 2020-11-16T20:22:52.183 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify cached snaps = True 2020-11-16T20:22:52.183 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify missing on start = True 2020-11-16T20:22:52.183 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2020-11-16T20:22:52.183 INFO:teuthology.orchestra.run.smithi092.stdout: osd max backfills = 3 2020-11-16T20:22:52.183 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2020-11-16T20:22:52.183 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2020-11-16T20:22:52.184 INFO:teuthology.orchestra.run.smithi092.stdout: osd op queue = debug_random 2020-11-16T20:22:52.184 INFO:teuthology.orchestra.run.smithi092.stdout: osd op queue cut off = debug_random 2020-11-16T20:22:52.184 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub min interval = 60 2020-11-16T20:22:52.184 INFO:teuthology.orchestra.run.smithi092.stdout: osd shutdown pgref assert = True 2020-11-16T20:22:52.184 INFO:teuthology.orchestra.run.smithi092.stdout: osd snap trim sleep = 2 2020-11-16T20:22:52.184 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.184 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2020-11-16T20:22:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2020-11-16T20:22:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2020-11-16T20:22:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2020-11-16T20:22:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2020-11-16T20:22:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2020-11-16T20:22:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2020-11-16T20:22:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2020-11-16T20:22:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.186 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2020-11-16T20:22:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2020-11-16T20:22:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2020-11-16T20:22:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2020-11-16T20:22:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2020-11-16T20:22:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2020-11-16T20:22:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2020-11-16T20:22:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2020-11-16T20:22:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2020-11-16T20:22:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2020-11-16T20:22:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2020-11-16T20:22:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: mon min osdmap epochs = 50 2020-11-16T20:22:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: mon osdmap full prune interval = 2 2020-11-16T20:22:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: mon osdmap full prune min = 15 2020-11-16T20:22:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: mon osdmap full prune txsize = 2 2020-11-16T20:22:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: paxos service trim min = 10 2020-11-16T20:22:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:22:52.188 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2020-11-16T20:22:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2020-11-16T20:22:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2020-11-16T20:22:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2020-11-16T20:22:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-16T20:22:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-16T20:22:52.189 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2020-11-16T20:22:52.189 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2020-11-16T20:22:52.189 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2020-11-16T20:22:52.190 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-16T20:22:52.190 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-16T20:22:52.280 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-16T20:22:52.281 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:52.281 INFO:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-16T20:22:52.284 INFO:teuthology.orchestra.run.smithi092.stderr:+ dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-16T20:22:52.285 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.] 2020-11-16T20:22:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCb37JfU0iPMxAAyPoH7yOdjBiriR7UdzyUBg== 2020-11-16T20:22:52.286 INFO:teuthology.orchestra.run.smithi092.stdout:[client.admin] 2020-11-16T20:22:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCc37JfgUnqEBAAgNdQOMEizwQRP/uvqkVTFQ== 2020-11-16T20:22:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: caps mds = "allow *" 2020-11-16T20:22:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: caps mgr = "allow *" 2020-11-16T20:22:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: caps mon = "allow *" 2020-11-16T20:22:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: caps osd = "allow *" 2020-11-16T20:22:52.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:22:52.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:22:52.287 INFO:teuthology.orchestra.run.smithi092.stderr:206 bytes copied, 7.4226e-05 s, 2.8 MB/s 2020-11-16T20:22:52.287 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:52.288 INFO:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-16T20:22:52.333 INFO:teuthology.orchestra.run.smithi092.stderr:+ dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-16T20:22:52.334 INFO:teuthology.orchestra.run.smithi092.stdout: �?ȤtO�M�V 2020-11-16T20:22:52.335 INFO:teuthology.orchestra.run.smithi092.stdout:�����߲_~� 2020-11-16T20:22:52.335 INFO:teuthology.orchestra.run.smithi092.stdout:�߲_~� 2020-11-16T20:22:52.335 INFO:teuthology.orchestra.run.smithi092.stdout:�aXa �\��\bXb � �� cXc �\��\abc 2020-11-16T20:22:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:22:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:22:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:409 bytes copied, 0.000154269 s, 2.7 MB/s 2020-11-16T20:22:52.336 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2020-11-16T20:22:52.336 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:52.336 INFO:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T20:22:52.337 INFO:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T20:22:52.381 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T20:22:52.390 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:22:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:22:52.391 INFO:teuthology.orchestra.run.smithi092.stderr:206 bytes copied, 0.000120817 s, 1.7 MB/s 2020-11-16T20:22:52.392 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T20:22:52.401 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:22:52.402 INFO:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:22:52.449 INFO:teuthology.orchestra.run.smithi092.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:22:52.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:22:52.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:22:52.451 INFO:teuthology.orchestra.run.smithi092.stderr:409 bytes copied, 0.000122543 s, 3.3 MB/s 2020-11-16T20:22:52.451 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T20:22:52.452 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:22:52.452 INFO:teuthology.orchestra.run.smithi013:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T20:22:52.452 INFO:teuthology.orchestra.run.smithi013:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T20:22:52.457 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-16T20:22:52.466 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:22:52.467 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:22:52.467 INFO:teuthology.orchestra.run.smithi013.stderr:206 bytes copied, 7.3273e-05 s, 2.8 MB/s 2020-11-16T20:22:52.467 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-16T20:22:52.476 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:22:52.477 INFO:teuthology.orchestra.run.smithi013:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:22:52.524 INFO:teuthology.orchestra.run.smithi013.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:22:52.526 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:22:52.526 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:22:52.526 INFO:teuthology.orchestra.run.smithi013.stderr:409 bytes copied, 7.602e-05 s, 5.4 MB/s 2020-11-16T20:22:52.526 INFO:tasks.ceph:Setting up mon nodes... 2020-11-16T20:22:52.527 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-16T20:22:52.527 INFO:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-16T20:22:52.572 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-16T20:22:52.574 INFO:teuthology.orchestra.run.smithi013:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-16T20:22:52.615 INFO:teuthology.orchestra.run.smithi013.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-16T20:22:52.617 INFO:tasks.ceph:Setting up mds nodes... 2020-11-16T20:22:52.618 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-16T20:22:52.618 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-16T20:22:52.653 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-16T20:22:52.664 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-11-16T20:22:52.700 INFO:teuthology.orchestra.run.smithi013.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-11-16T20:22:52.713 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-16T20:22:52.713 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi013.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-16T20:22:52.713 INFO:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-16T20:22:52.728 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T20:22:52.729 INFO:tasks.ceph:role: osd.0 2020-11-16T20:22:52.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2020-11-16T20:22:52.729 INFO:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T20:22:53.241 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T20:22:53.241 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T20:22:53.241 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T20:22:53.241 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T20:22:53.242 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2020-11-16T20:22:53.242 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T20:22:53.242 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T20:22:53.242 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T20:22:53.242 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T20:22:53.243 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2020-11-16T20:22:53.243 INFO:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-16T20:22:53.318 INFO:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-16T20:22:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T20:22:53.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T20:22:53.329 INFO:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-16T20:22:53.387 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T20:22:53.387 INFO:tasks.ceph:role: osd.1 2020-11-16T20:22:53.388 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2020-11-16T20:22:53.388 INFO:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-16T20:22:53.904 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T20:22:53.905 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T20:22:53.905 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T20:22:53.905 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T20:22:53.905 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2020-11-16T20:22:53.905 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T20:22:53.905 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T20:22:53.906 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T20:22:53.906 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T20:22:53.907 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2020-11-16T20:22:53.907 INFO:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-11-16T20:22:53.980 INFO:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-16T20:22:53.991 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T20:22:53.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T20:22:53.992 INFO:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-16T20:22:54.046 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T20:22:54.046 INFO:tasks.ceph:role: osd.2 2020-11-16T20:22:54.046 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2020-11-16T20:22:58.166 INFO:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-16T20:22:58.653 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T20:22:58.653 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T20:22:58.654 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T20:22:58.654 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T20:22:58.654 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2020-11-16T20:22:58.654 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T20:22:58.654 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T20:22:58.654 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T20:22:58.654 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T20:22:58.655 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2020-11-16T20:22:58.656 INFO:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-11-16T20:22:58.710 INFO:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-16T20:22:58.721 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T20:22:58.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T20:22:58.722 INFO:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-11-16T20:22:58.780 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-16T20:22:58.781 INFO:tasks.ceph:role: osd.3 2020-11-16T20:22:58.781 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2020-11-16T20:22:58.781 INFO:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-16T20:22:59.300 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T20:22:59.300 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T20:22:59.300 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T20:22:59.300 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T20:22:59.300 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2020-11-16T20:22:59.301 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T20:22:59.301 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T20:22:59.301 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T20:22:59.301 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T20:22:59.302 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2020-11-16T20:22:59.302 INFO:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-11-16T20:22:59.348 INFO:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-11-16T20:22:59.356 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T20:22:59.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T20:22:59.358 INFO:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:22:59.461 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:22:59.463+0000 7f32b1472d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-16T20:22:59.464 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:22:59.467+0000 7f32b1472d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-16T20:22:59.464 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:22:59.467+0000 7f32b1472d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T20:22:59.464 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:22:59.467+0000 7f32b1472d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T20:22:59.464 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:22:59.467+0000 7f32b1472d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-16T20:22:59.473 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:22:59.479+0000 7f32b1472d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-16T20:23:02.434 INFO:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-16T20:23:02.448 INFO:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:02.530 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:02.535+0000 7f2b35f40d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-16T20:23:02.531 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:02.535+0000 7f2b35f40d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-16T20:23:02.531 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:02.535+0000 7f2b35f40d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T20:23:02.531 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:02.535+0000 7f2b35f40d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T20:23:02.531 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:02.535+0000 7f2b35f40d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-16T20:23:02.532 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:02.535+0000 7f2b35f40d80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-16T20:23:05.481 INFO:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-16T20:23:05.540 INFO:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:05.618 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:05.623+0000 7f21e82e5d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-16T20:23:05.619 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:05.623+0000 7f21e82e5d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-16T20:23:05.619 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:05.623+0000 7f21e82e5d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T20:23:05.620 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:05.623+0000 7f21e82e5d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T20:23:05.620 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:05.623+0000 7f21e82e5d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-16T20:23:05.620 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:05.623+0000 7f21e82e5d80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-16T20:23:08.568 INFO:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-16T20:23:08.624 INFO:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:08.703 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:08.707+0000 7fd0ffce5d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-11-16T20:23:08.703 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:08.707+0000 7fd0ffce5d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-11-16T20:23:08.703 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:08.707+0000 7fd0ffce5d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T20:23:08.703 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:08.707+0000 7fd0ffce5d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T20:23:08.704 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:08.707+0000 7fd0ffce5d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-16T20:23:08.704 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:08.707+0000 7fd0ffce5d80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-11-16T20:23:11.656 INFO:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-11-16T20:23:11.712 INFO:teuthology.orchestra.run.smithi013:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-11-16T20:23:11.728 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T20:23:11.728 INFO:tasks.ceph:role: osd.4 2020-11-16T20:23:11.728 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T20:23:11.728 INFO:teuthology.orchestra.run.smithi013:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-16T20:23:11.810 INFO:teuthology.orchestra.run.smithi013.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T20:23:11.810 INFO:teuthology.orchestra.run.smithi013.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T20:23:11.810 INFO:teuthology.orchestra.run.smithi013.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T20:23:11.810 INFO:teuthology.orchestra.run.smithi013.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T20:23:11.811 INFO:teuthology.orchestra.run.smithi013.stdout: = sunit=0 swidth=0 blks 2020-11-16T20:23:11.811 INFO:teuthology.orchestra.run.smithi013.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T20:23:11.811 INFO:teuthology.orchestra.run.smithi013.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T20:23:11.811 INFO:teuthology.orchestra.run.smithi013.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T20:23:11.811 INFO:teuthology.orchestra.run.smithi013.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T20:23:11.812 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi013.front.sepia.ceph.com -o noatime 2020-11-16T20:23:11.812 INFO:teuthology.orchestra.run.smithi013:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-11-16T20:23:11.875 INFO:teuthology.orchestra.run.smithi013:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-11-16T20:23:11.926 INFO:teuthology.orchestra.run.smithi013.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T20:23:11.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T20:23:11.928 INFO:teuthology.orchestra.run.smithi013:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-11-16T20:23:11.981 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T20:23:11.981 INFO:tasks.ceph:role: osd.5 2020-11-16T20:23:11.981 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T20:23:11.982 INFO:teuthology.orchestra.run.smithi013:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-16T20:23:12.065 INFO:teuthology.orchestra.run.smithi013.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T20:23:12.065 INFO:teuthology.orchestra.run.smithi013.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T20:23:12.066 INFO:teuthology.orchestra.run.smithi013.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T20:23:12.066 INFO:teuthology.orchestra.run.smithi013.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T20:23:12.066 INFO:teuthology.orchestra.run.smithi013.stdout: = sunit=0 swidth=0 blks 2020-11-16T20:23:12.066 INFO:teuthology.orchestra.run.smithi013.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T20:23:12.066 INFO:teuthology.orchestra.run.smithi013.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T20:23:12.066 INFO:teuthology.orchestra.run.smithi013.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T20:23:12.067 INFO:teuthology.orchestra.run.smithi013.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T20:23:12.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi013.front.sepia.ceph.com -o noatime 2020-11-16T20:23:12.068 INFO:teuthology.orchestra.run.smithi013:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-11-16T20:23:12.131 INFO:teuthology.orchestra.run.smithi013:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-11-16T20:23:12.183 INFO:teuthology.orchestra.run.smithi013.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T20:23:12.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T20:23:12.184 INFO:teuthology.orchestra.run.smithi013:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-11-16T20:23:12.238 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T20:23:16.410 INFO:tasks.ceph:role: osd.6 2020-11-16T20:23:16.410 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T20:23:16.411 INFO:teuthology.orchestra.run.smithi013:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-16T20:23:16.455 INFO:teuthology.orchestra.run.smithi013.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T20:23:16.455 INFO:teuthology.orchestra.run.smithi013.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T20:23:16.456 INFO:teuthology.orchestra.run.smithi013.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T20:23:16.456 INFO:teuthology.orchestra.run.smithi013.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T20:23:16.456 INFO:teuthology.orchestra.run.smithi013.stdout: = sunit=0 swidth=0 blks 2020-11-16T20:23:16.456 INFO:teuthology.orchestra.run.smithi013.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T20:23:16.456 INFO:teuthology.orchestra.run.smithi013.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T20:23:16.457 INFO:teuthology.orchestra.run.smithi013.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T20:23:16.457 INFO:teuthology.orchestra.run.smithi013.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T20:23:16.458 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi013.front.sepia.ceph.com -o noatime 2020-11-16T20:23:16.458 INFO:teuthology.orchestra.run.smithi013:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-11-16T20:23:16.519 INFO:teuthology.orchestra.run.smithi013:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-11-16T20:23:16.568 INFO:teuthology.orchestra.run.smithi013.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T20:23:16.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T20:23:16.570 INFO:teuthology.orchestra.run.smithi013:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-11-16T20:23:16.625 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-16T20:23:16.625 INFO:tasks.ceph:role: osd.7 2020-11-16T20:23:16.625 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi013.front.sepia.ceph.com 2020-11-16T20:23:16.626 INFO:teuthology.orchestra.run.smithi013:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-16T20:23:16.710 INFO:teuthology.orchestra.run.smithi013.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-16T20:23:16.710 INFO:teuthology.orchestra.run.smithi013.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-16T20:23:16.710 INFO:teuthology.orchestra.run.smithi013.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-16T20:23:16.710 INFO:teuthology.orchestra.run.smithi013.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-16T20:23:16.710 INFO:teuthology.orchestra.run.smithi013.stdout: = sunit=0 swidth=0 blks 2020-11-16T20:23:16.711 INFO:teuthology.orchestra.run.smithi013.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-16T20:23:16.711 INFO:teuthology.orchestra.run.smithi013.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-16T20:23:16.711 INFO:teuthology.orchestra.run.smithi013.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-16T20:23:16.711 INFO:teuthology.orchestra.run.smithi013.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-16T20:23:16.712 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi013.front.sepia.ceph.com -o noatime 2020-11-16T20:23:16.712 INFO:teuthology.orchestra.run.smithi013:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-11-16T20:23:16.774 INFO:teuthology.orchestra.run.smithi013:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-11-16T20:23:16.825 INFO:teuthology.orchestra.run.smithi013.stderr:sudo: /sbin/restorecon: command not found 2020-11-16T20:23:16.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-16T20:23:16.827 INFO:teuthology.orchestra.run.smithi013:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:16.930 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:16.926+0000 7fe312ed7d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-11-16T20:23:16.930 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:16.926+0000 7fe312ed7d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-11-16T20:23:16.931 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:16.926+0000 7fe312ed7d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T20:23:16.931 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:16.926+0000 7fe312ed7d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T20:23:16.931 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:16.926+0000 7fe312ed7d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-16T20:23:16.931 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:16.926+0000 7fe312ed7d80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-11-16T20:23:19.893 INFO:teuthology.orchestra.run.smithi013:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-11-16T20:23:19.908 INFO:teuthology.orchestra.run.smithi013:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:19.990 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:19.986+0000 7f1c59939d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-11-16T20:23:19.991 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:19.986+0000 7f1c59939d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-11-16T20:23:19.991 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:19.986+0000 7f1c59939d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T20:23:19.991 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:19.986+0000 7f1c59939d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T20:23:19.991 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:19.986+0000 7f1c59939d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-16T20:23:19.992 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:19.986+0000 7f1c59939d80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-11-16T20:23:22.952 INFO:teuthology.orchestra.run.smithi013:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-11-16T20:23:23.012 INFO:teuthology.orchestra.run.smithi013:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:23.094 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:23.090+0000 7fef99974d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-11-16T20:23:23.095 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:23.090+0000 7fef99974d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-11-16T20:23:23.095 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:23.090+0000 7fef99974d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T20:23:23.095 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:23.090+0000 7fef99974d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T20:23:23.095 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:23.090+0000 7fef99974d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-16T20:23:23.096 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:23.090+0000 7fef99974d80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-11-16T20:23:26.047 INFO:teuthology.orchestra.run.smithi013:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-11-16T20:23:26.104 INFO:teuthology.orchestra.run.smithi013:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:26.186 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:26.182+0000 7fb2d56eed80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-11-16T20:23:26.187 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:26.182+0000 7fb2d56eed80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-11-16T20:23:26.187 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:26.182+0000 7fb2d56eed80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T20:23:26.187 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:26.182+0000 7fb2d56eed80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T20:23:26.187 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:26.182+0000 7fb2d56eed80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-16T20:23:26.188 INFO:teuthology.orchestra.run.smithi013.stderr:2020-11-16T20:23:26.182+0000 7fb2d56eed80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-11-16T20:23:29.144 INFO:teuthology.orchestra.run.smithi013:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-11-16T20:23:29.201 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-16T20:23:29.202 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:29.202 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-16T20:23:29.207 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-16T20:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2020-11-16T20:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCc37JfAHxtIhAAwUoxW3ZxvXP+HShfKHpqjA== 2020-11-16T20:23:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:56 bytes copied, 0.000171463 s, 327 kB/s 2020-11-16T20:23:29.218 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:29.218 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-16T20:23:29.264 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-16T20:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2020-11-16T20:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCj37JfGxXVGxAAuz+PGiFl258rofSlpv+eug== 2020-11-16T20:23:29.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:29.274 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:29.274 INFO:teuthology.orchestra.run.smithi092.stderr:56 bytes copied, 0.000160556 s, 349 kB/s 2020-11-16T20:23:29.275 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:29.275 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-16T20:23:29.324 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-16T20:23:29.333 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2020-11-16T20:23:29.333 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCm37JfUBL1HxAAParkZVpmvUSDrQpUuIZ8Sw== 2020-11-16T20:23:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:29.334 INFO:teuthology.orchestra.run.smithi092.stderr:56 bytes copied, 0.000155673 s, 360 kB/s 2020-11-16T20:23:29.334 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:29.335 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-16T20:23:29.380 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-16T20:23:29.387 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2020-11-16T20:23:29.388 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCp37Jflz88JRAA0jv9186efzBLLKGP+sAGTQ== 2020-11-16T20:23:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:56 bytes copied, 0.000160314 s, 349 kB/s 2020-11-16T20:23:29.389 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:29.389 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-16T20:23:29.436 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-16T20:23:29.445 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2020-11-16T20:23:29.445 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCs37Jf/9M8KhAACcmoalhl1/OPrA6nPAEDHA== 2020-11-16T20:23:29.445 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:29.445 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:29.445 INFO:teuthology.orchestra.run.smithi092.stderr:56 bytes copied, 0.000156763 s, 357 kB/s 2020-11-16T20:23:29.446 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:29.447 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-16T20:23:29.451 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-16T20:23:29.460 INFO:teuthology.orchestra.run.smithi013.stdout:[mgr.x] 2020-11-16T20:23:29.461 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCc37JfasGXJBAAs05Zus4DgqlvrXeYB8kLuQ== 2020-11-16T20:23:29.461 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:29.461 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:29.461 INFO:teuthology.orchestra.run.smithi013.stderr:56 bytes copied, 0.000179324 s, 312 kB/s 2020-11-16T20:23:29.462 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:29.462 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-16T20:23:29.507 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-16T20:23:29.517 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.4] 2020-11-16T20:23:29.517 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQC037JfZgVcNxAAUHpYNfqSLIicTel8EXX/hg== 2020-11-16T20:23:29.517 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:29.517 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:29.517 INFO:teuthology.orchestra.run.smithi013.stderr:56 bytes copied, 0.000164231 s, 341 kB/s 2020-11-16T20:23:29.518 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:29.518 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-16T20:23:29.563 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-16T20:23:29.572 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.5] 2020-11-16T20:23:29.573 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQC337Jfc5P0OhAAyVRfimw0c65ed9yeI6KvLA== 2020-11-16T20:23:29.573 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:29.573 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:29.573 INFO:teuthology.orchestra.run.smithi013.stderr:56 bytes copied, 0.000172757 s, 324 kB/s 2020-11-16T20:23:29.574 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:29.574 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-16T20:23:29.619 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-16T20:23:29.629 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.6] 2020-11-16T20:23:29.629 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQC737Jf/OSNBRAA/iEu79y+XEMyStPoD/eHMQ== 2020-11-16T20:23:29.629 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:29.629 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:29.630 INFO:teuthology.orchestra.run.smithi013.stderr:56 bytes copied, 0.000170603 s, 328 kB/s 2020-11-16T20:23:29.630 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:29.631 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-16T20:23:29.676 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-16T20:23:29.685 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.7] 2020-11-16T20:23:29.685 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQC+37JfGDQOCxAA7uOyOFLxu3wck4dl9E9KBA== 2020-11-16T20:23:29.685 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:29.685 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:29.685 INFO:teuthology.orchestra.run.smithi013.stderr:56 bytes copied, 0.000212493 s, 264 kB/s 2020-11-16T20:23:29.686 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:29.686 INFO:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-16T20:23:29.691 INFO:teuthology.orchestra.run.smithi092.stderr:+ dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-16T20:23:29.693 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2020-11-16T20:23:29.693 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCc37Jf9HZKJxAAPn63Ad99FzGk3UXN0tK6/A== 2020-11-16T20:23:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:59 bytes copied, 0.000161337 s, 366 kB/s 2020-11-16T20:23:29.694 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:29.694 INFO:teuthology.orchestra.run.smithi013:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2020-11-16T20:23:29.732 INFO:teuthology.orchestra.run.smithi013.stderr:+ dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2020-11-16T20:23:29.733 INFO:teuthology.orchestra.run.smithi013.stdout:[client.1] 2020-11-16T20:23:29.733 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCc37JfWeWkKRAAjp/JgBQUm3lmWYo84AtBkQ== 2020-11-16T20:23:29.733 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:29.733 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:29.734 INFO:teuthology.orchestra.run.smithi013.stderr:59 bytes copied, 0.000169887 s, 347 kB/s 2020-11-16T20:23:29.734 INFO:tasks.ceph:Adding keys to all mons... 2020-11-16T20:23:29.734 INFO:teuthology.orchestra.run.smithi013:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-16T20:23:29.777 INFO:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-16T20:23:29.789 INFO:teuthology.orchestra.run.smithi013.stdout:[mgr.y] 2020-11-16T20:23:29.789 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCc37JfAHxtIhAAwUoxW3ZxvXP+HShfKHpqjA== 2020-11-16T20:23:29.789 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.0] 2020-11-16T20:23:29.789 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCj37JfGxXVGxAAuz+PGiFl258rofSlpv+eug== 2020-11-16T20:23:29.790 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.1] 2020-11-16T20:23:29.790 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCm37JfUBL1HxAAParkZVpmvUSDrQpUuIZ8Sw== 2020-11-16T20:23:29.790 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.2] 2020-11-16T20:23:29.790 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCp37Jflz88JRAA0jv9186efzBLLKGP+sAGTQ== 2020-11-16T20:23:29.790 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.3] 2020-11-16T20:23:29.790 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCs37Jf/9M8KhAACcmoalhl1/OPrA6nPAEDHA== 2020-11-16T20:23:29.790 INFO:teuthology.orchestra.run.smithi013.stdout:[mgr.x] 2020-11-16T20:23:29.791 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCc37JfasGXJBAAs05Zus4DgqlvrXeYB8kLuQ== 2020-11-16T20:23:29.791 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.4] 2020-11-16T20:23:29.791 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQC037JfZgVcNxAAUHpYNfqSLIicTel8EXX/hg== 2020-11-16T20:23:29.791 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.5] 2020-11-16T20:23:29.791 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQC337Jfc5P0OhAAyVRfimw0c65ed9yeI6KvLA== 2020-11-16T20:23:29.791 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.6] 2020-11-16T20:23:29.791 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQC737Jf/OSNBRAA/iEu79y+XEMyStPoD/eHMQ== 2020-11-16T20:23:29.792 INFO:teuthology.orchestra.run.smithi013.stdout:[osd.7] 2020-11-16T20:23:29.792 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQC+37JfGDQOCxAA7uOyOFLxu3wck4dl9E9KBA== 2020-11-16T20:23:29.792 INFO:teuthology.orchestra.run.smithi013.stdout:[client.0] 2020-11-16T20:23:29.792 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCc37Jf9HZKJxAAPn63Ad99FzGk3UXN0tK6/A== 2020-11-16T20:23:29.792 INFO:teuthology.orchestra.run.smithi013.stdout:[client.1] 2020-11-16T20:23:29.792 INFO:teuthology.orchestra.run.smithi013.stdout: key = AQCc37JfWeWkKRAAjp/JgBQUm3lmWYo84AtBkQ== 2020-11-16T20:23:29.797 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2020-11-16T20:23:29.797 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCc37JfAHxtIhAAwUoxW3ZxvXP+HShfKHpqjA== 2020-11-16T20:23:29.797 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2020-11-16T20:23:29.797 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCj37JfGxXVGxAAuz+PGiFl258rofSlpv+eug== 2020-11-16T20:23:29.798 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2020-11-16T20:23:29.798 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCm37JfUBL1HxAAParkZVpmvUSDrQpUuIZ8Sw== 2020-11-16T20:23:29.798 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2020-11-16T20:23:29.798 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCp37Jflz88JRAA0jv9186efzBLLKGP+sAGTQ== 2020-11-16T20:23:29.798 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2020-11-16T20:23:29.798 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCs37Jf/9M8KhAACcmoalhl1/OPrA6nPAEDHA== 2020-11-16T20:23:29.798 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2020-11-16T20:23:29.799 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCc37JfasGXJBAAs05Zus4DgqlvrXeYB8kLuQ== 2020-11-16T20:23:29.799 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2020-11-16T20:23:29.799 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC037JfZgVcNxAAUHpYNfqSLIicTel8EXX/hg== 2020-11-16T20:23:29.799 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2020-11-16T20:23:29.799 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC337Jfc5P0OhAAyVRfimw0c65ed9yeI6KvLA== 2020-11-16T20:23:29.799 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2020-11-16T20:23:29.799 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC737Jf/OSNBRAA/iEu79y+XEMyStPoD/eHMQ== 2020-11-16T20:23:29.800 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2020-11-16T20:23:29.800 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC+37JfGDQOCxAA7uOyOFLxu3wck4dl9E9KBA== 2020-11-16T20:23:29.800 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2020-11-16T20:23:29.800 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCc37Jf9HZKJxAAPn63Ad99FzGk3UXN0tK6/A== 2020-11-16T20:23:29.800 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2020-11-16T20:23:29.800 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCc37JfWeWkKRAAjp/JgBQUm3lmWYo84AtBkQ== 2020-11-16T20:23:29.801 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T20:23:29.841 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T20:23:29.882 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:29.925 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:29.966 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.009 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.052 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.093 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.137 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.177 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.222 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T20:23:30.261 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-16T20:23:30.308 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.345 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.388 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.431 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.477 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.522 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.566 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.608 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-16T20:23:30.650 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T20:23:30.693 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T20:23:30.739 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T20:23:30.741 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-16T20:23:30.825 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-16T20:23:30.825 INFO:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-16T20:23:30.834 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T20:23:30.934 INFO:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-16T20:23:30.946 INFO:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-16T20:23:31.002 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T20:23:31.115 INFO:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-16T20:23:31.131 INFO:teuthology.orchestra.run.smithi013:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-16T20:23:31.146 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-16T20:23:31.244 INFO:teuthology.orchestra.run.smithi013:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-16T20:23:31.258 INFO:teuthology.orchestra.run.smithi013:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:31.305 INFO:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-16T20:23:31.311 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-16T20:23:31.311 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-16T20:23:31.312 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-16T20:23:31.353 INFO:tasks.ceph.mon.a:Started 2020-11-16T20:23:31.354 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-16T20:23:31.354 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-16T20:23:31.359 INFO:tasks.ceph.mon.c:Started 2020-11-16T20:23:31.360 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-16T20:23:31.360 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-16T20:23:31.362 INFO:tasks.ceph.mon.b:Started 2020-11-16T20:23:31.362 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-16T20:23:31.363 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-16T20:23:31.363 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-16T20:23:31.365 INFO:tasks.ceph.mgr.y:Started 2020-11-16T20:23:31.365 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-16T20:23:31.366 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-16T20:23:31.368 INFO:tasks.ceph.mgr.x:Started 2020-11-16T20:23:31.368 INFO:tasks.ceph:Setting crush tunables to default 2020-11-16T20:23:31.368 INFO:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2020-11-16T20:23:31.832 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2020-11-16T20:23:31.846 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-16T20:23:31.847 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:31.847 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-16T20:23:31.851 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-16T20:23:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:041f4de8-6113-42d0-89ef-3ff955e898a6 2020-11-16T20:23:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:37 bytes copied, 0.000105453 s, 351 kB/s 2020-11-16T20:23:31.860 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:31.860 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-16T20:23:31.903 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-16T20:23:31.911 INFO:teuthology.orchestra.run.smithi092.stdout:cf43191f-7015-40ca-a363-9892eed40e46 2020-11-16T20:23:31.911 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:31.911 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:31.911 INFO:teuthology.orchestra.run.smithi092.stderr:37 bytes copied, 0.000159916 s, 231 kB/s 2020-11-16T20:23:31.912 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:31.912 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-16T20:23:31.959 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-16T20:23:31.967 INFO:teuthology.orchestra.run.smithi092.stdout:e42e4433-f9d2-4e5d-9cff-2a1ef11c6736 2020-11-16T20:23:31.967 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:31.967 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:31.967 INFO:teuthology.orchestra.run.smithi092.stderr:37 bytes copied, 0.000170694 s, 217 kB/s 2020-11-16T20:23:31.968 INFO:teuthology.orchestra.run.smithi092:> set -ex 2020-11-16T20:23:31.968 INFO:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-16T20:23:32.016 INFO:teuthology.orchestra.run.smithi092.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-16T20:23:32.024 INFO:teuthology.orchestra.run.smithi092.stdout:2b9e1ff5-7b30-4565-bc03-527cf9cee5dd 2020-11-16T20:23:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2020-11-16T20:23:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2020-11-16T20:23:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:37 bytes copied, 0.000161817 s, 229 kB/s 2020-11-16T20:23:32.025 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:32.025 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-16T20:23:32.030 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-16T20:23:32.039 INFO:teuthology.orchestra.run.smithi013.stdout:e75baa38-b785-4bde-9681-fe474fbc202d 2020-11-16T20:23:32.040 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:32.040 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:32.040 INFO:teuthology.orchestra.run.smithi013.stderr:37 bytes copied, 0.000216753 s, 171 kB/s 2020-11-16T20:23:32.041 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:32.041 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-16T20:23:32.087 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-16T20:23:32.096 INFO:teuthology.orchestra.run.smithi013.stdout:a45307aa-ecba-42fc-9a72-5b2e4b1e3117 2020-11-16T20:23:32.096 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:32.097 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:32.097 INFO:teuthology.orchestra.run.smithi013.stderr:37 bytes copied, 0.000187995 s, 197 kB/s 2020-11-16T20:23:32.098 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:32.098 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-16T20:23:32.143 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-16T20:23:32.151 INFO:teuthology.orchestra.run.smithi013.stdout:9c6a25e5-0753-4079-9873-09a212a61f51 2020-11-16T20:23:32.151 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:32.151 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:32.151 INFO:teuthology.orchestra.run.smithi013.stderr:37 bytes copied, 0.000160007 s, 231 kB/s 2020-11-16T20:23:32.152 INFO:teuthology.orchestra.run.smithi013:> set -ex 2020-11-16T20:23:32.152 INFO:teuthology.orchestra.run.smithi013:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-16T20:23:32.199 INFO:teuthology.orchestra.run.smithi013.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-16T20:23:32.207 INFO:teuthology.orchestra.run.smithi013.stdout:2cb56fe5-4a39-4599-8f46-0d452347ee33 2020-11-16T20:23:32.208 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records in 2020-11-16T20:23:32.208 INFO:teuthology.orchestra.run.smithi013.stderr:0+1 records out 2020-11-16T20:23:32.208 INFO:teuthology.orchestra.run.smithi013.stderr:37 bytes copied, 0.000165874 s, 223 kB/s 2020-11-16T20:23:32.209 INFO:teuthology.orchestra.run.smithi013:> sudo ceph --cluster ceph osd new 041f4de8-6113-42d0-89ef-3ff955e898a6 0 2020-11-16T20:23:32.542 INFO:teuthology.orchestra.run.smithi013.stdout:0 2020-11-16T20:23:32.555 INFO:teuthology.orchestra.run.smithi013:> sudo ceph --cluster ceph osd new cf43191f-7015-40ca-a363-9892eed40e46 1 2020-11-16T20:23:32.778 INFO:teuthology.orchestra.run.smithi013.stdout:1 2020-11-16T20:23:32.792 INFO:teuthology.orchestra.run.smithi013:> sudo ceph --cluster ceph osd new e42e4433-f9d2-4e5d-9cff-2a1ef11c6736 2 2020-11-16T20:23:33.014 INFO:teuthology.orchestra.run.smithi013.stdout:2 2020-11-16T20:23:33.028 INFO:teuthology.orchestra.run.smithi013:> sudo ceph --cluster ceph osd new 2b9e1ff5-7b30-4565-bc03-527cf9cee5dd 3 2020-11-16T20:23:33.253 INFO:teuthology.orchestra.run.smithi013.stdout:3 2020-11-16T20:23:33.266 INFO:teuthology.orchestra.run.smithi013:> sudo ceph --cluster ceph osd new e75baa38-b785-4bde-9681-fe474fbc202d 4 2020-11-16T20:23:33.488 INFO:teuthology.orchestra.run.smithi013.stdout:4 2020-11-16T20:23:33.500 INFO:teuthology.orchestra.run.smithi013:> sudo ceph --cluster ceph osd new a45307aa-ecba-42fc-9a72-5b2e4b1e3117 5 2020-11-16T20:23:33.732 INFO:teuthology.orchestra.run.smithi013.stdout:5 2020-11-16T20:23:33.746 INFO:teuthology.orchestra.run.smithi013:> sudo ceph --cluster ceph osd new 9c6a25e5-0753-4079-9873-09a212a61f51 6 2020-11-16T20:23:33.981 INFO:teuthology.orchestra.run.smithi013.stdout:6 2020-11-16T20:23:33.994 INFO:teuthology.orchestra.run.smithi013:> sudo ceph --cluster ceph osd new 2cb56fe5-4a39-4599-8f46-0d452347ee33 7 2020-11-16T20:23:34.217 INFO:teuthology.orchestra.run.smithi013.stdout:7 2020-11-16T20:23:34.230 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-16T20:23:34.231 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-16T20:23:34.233 INFO:tasks.ceph.osd.0:Started 2020-11-16T20:23:34.233 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-16T20:23:34.233 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-16T20:23:34.235 INFO:tasks.ceph.osd.1:Started 2020-11-16T20:23:34.236 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-16T20:23:34.236 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-16T20:23:34.238 INFO:tasks.ceph.osd.2:Started 2020-11-16T20:23:34.238 INFO:tasks.ceph.osd.3:Restarting daemon 2020-11-16T20:23:34.238 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-11-16T20:23:34.241 INFO:tasks.ceph.osd.3:Started 2020-11-16T20:23:34.242 INFO:tasks.ceph.osd.4:Restarting daemon 2020-11-16T20:23:34.242 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-11-16T20:23:34.244 INFO:tasks.ceph.osd.4:Started 2020-11-16T20:23:34.244 INFO:tasks.ceph.osd.5:Restarting daemon 2020-11-16T20:23:34.244 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-11-16T20:23:34.246 INFO:tasks.ceph.osd.5:Started 2020-11-16T20:23:34.246 INFO:tasks.ceph.osd.6:Restarting daemon 2020-11-16T20:23:34.247 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-11-16T20:23:34.248 INFO:tasks.ceph.osd.6:Started 2020-11-16T20:23:34.249 INFO:tasks.ceph.osd.7:Restarting daemon 2020-11-16T20:23:34.249 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-11-16T20:23:34.251 INFO:tasks.ceph.osd.7:Started 2020-11-16T20:23:34.251 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-16T20:23:34.610 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:23:34.615+0000 7ff2463f1d80 -1 Falling back to public interface 2020-11-16T20:23:34.621 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:23:34.618+0000 7feaf0f89d80 -1 Falling back to public interface 2020-11-16T20:23:34.649 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:23:34.646+0000 7f7399823d80 -1 Falling back to public interface 2020-11-16T20:23:34.854 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:23:34.859+0000 7f37e0c9fd80 -1 Falling back to public interface 2020-11-16T20:23:34.854 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:23:34.859+0000 7fbef7936d80 -1 Falling back to public interface 2020-11-16T20:23:34.865 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:23:34.871+0000 7f1e6d001d80 -1 Falling back to public interface 2020-11-16T20:23:34.881 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:23:34.878+0000 7f11fd5ded80 -1 Falling back to public interface 2020-11-16T20:23:34.882 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:23:34.878+0000 7f93892a0d80 -1 Falling back to public interface 2020-11-16T20:23:35.058 INFO:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-16T20:23:35.258 INFO:teuthology.misc.health.smithi092.stdout: 2020-11-16T20:23:35.258 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"3fc8a474-4fd1-4d15-a656-0a82bbfce204","created":"2020-11-16T20:23:31.699731+0000","modified":"2020-11-16T20:23:34.212316+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"041f4de8-6113-42d0-89ef-3ff955e898a6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cf43191f-7015-40ca-a363-9892eed40e46","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e42e4433-f9d2-4e5d-9cff-2a1ef11c6736","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2b9e1ff5-7b30-4565-bc03-527cf9cee5dd","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e75baa38-b785-4bde-9681-fe474fbc202d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a45307aa-ecba-42fc-9a72-5b2e4b1e3117","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9c6a25e5-0753-4079-9873-09a212a61f51","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2cb56fe5-4a39-4599-8f46-0d452347ee33","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T20:23:35.269 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-11-16T20:23:35.785 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:23:35.782+0000 7feaf0f89d80 -1 osd.5 0 log_to_monitors {default=true} 2020-11-16T20:23:35.786 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:23:35.791+0000 7ff2463f1d80 -1 osd.0 0 log_to_monitors {default=true} 2020-11-16T20:23:36.073 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:23:36.070+0000 7f7399823d80 -1 osd.6 0 log_to_monitors {default=true} 2020-11-16T20:23:36.269 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:23:36.266+0000 7f11fd5ded80 -1 osd.7 0 log_to_monitors {default=true} 2020-11-16T20:23:36.291 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:23:36.295+0000 7fbef7936d80 -1 osd.3 0 log_to_monitors {default=true} 2020-11-16T20:23:36.319 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:23:36.314+0000 7f93892a0d80 -1 osd.4 0 log_to_monitors {default=true} 2020-11-16T20:23:36.320 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:23:36.323+0000 7f37e0c9fd80 -1 osd.1 0 log_to_monitors {default=true} 2020-11-16T20:23:36.334 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:23:36.339+0000 7f1e6d001d80 -1 osd.2 0 log_to_monitors {default=true} 2020-11-16T20:23:37.638 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:23:37.634+0000 7f938076a700 -1 osd.4 0 waiting for initial osdmap 2020-11-16T20:23:37.640 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:23:37.638+0000 7feae8453700 -1 osd.5 0 waiting for initial osdmap 2020-11-16T20:23:37.640 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:23:37.647+0000 7f1e644cb700 -1 osd.2 0 waiting for initial osdmap 2020-11-16T20:23:37.640 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:23:37.643+0000 7fbef0603700 -1 osd.3 0 waiting for initial osdmap 2020-11-16T20:23:37.641 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:23:37.638+0000 7f7390ced700 -1 osd.6 0 waiting for initial osdmap 2020-11-16T20:23:37.641 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:23:37.638+0000 7f11f4aa8700 -1 osd.7 0 waiting for initial osdmap 2020-11-16T20:23:37.641 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:23:37.647+0000 7ff23d8bb700 -1 osd.0 0 waiting for initial osdmap 2020-11-16T20:23:37.642 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:23:37.647+0000 7f37d8169700 -1 osd.1 0 waiting for initial osdmap 2020-11-16T20:23:42.077 INFO:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-16T20:23:42.365 INFO:teuthology.misc.health.smithi092.stdout: 2020-11-16T20:23:42.365 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":15,"fsid":"3fc8a474-4fd1-4d15-a656-0a82bbfce204","created":"2020-11-16T20:23:31.699731+0000","modified":"2020-11-16T20:23:40.706015+0000","last_up_change":"2020-11-16T20:23:38.624506+0000","last_in_change":"2020-11-16T20:23:38.624506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T20:23:35.658847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"041f4de8-6113-42d0-89ef-3ff955e898a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":12741},{"type":"v1","addr":"172.21.15.92:6801","nonce":12741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":12741},{"type":"v1","addr":"172.21.15.92:6803","nonce":12741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":12741},{"type":"v1","addr":"172.21.15.92:6807","nonce":12741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":12741},{"type":"v1","addr":"172.21.15.92:6805","nonce":12741}]},"public_addr":"172.21.15.92:6801/12741","cluster_addr":"172.21.15.92:6803/12741","heartbeat_back_addr":"172.21.15.92:6807/12741","heartbeat_front_addr":"172.21.15.92:6805/12741","state":["exists","up"]},{"osd":1,"uuid":"cf43191f-7015-40ca-a363-9892eed40e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":12731},{"type":"v1","addr":"172.21.15.92:6810","nonce":12731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":12731},{"type":"v1","addr":"172.21.15.92:6813","nonce":12731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":12731},{"type":"v1","addr":"172.21.15.92:6822","nonce":12731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":12731},{"type":"v1","addr":"172.21.15.92:6817","nonce":12731}]},"public_addr":"172.21.15.92:6810/12731","cluster_addr":"172.21.15.92:6813/12731","heartbeat_back_addr":"172.21.15.92:6822/12731","heartbeat_front_addr":"172.21.15.92:6817/12731","state":["exists","up"]},{"osd":2,"uuid":"e42e4433-f9d2-4e5d-9cff-2a1ef11c6736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":12732},{"type":"v1","addr":"172.21.15.92:6825","nonce":12732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":12732},{"type":"v1","addr":"172.21.15.92:6827","nonce":12732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":12732},{"type":"v1","addr":"172.21.15.92:6831","nonce":12732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":12732},{"type":"v1","addr":"172.21.15.92:6829","nonce":12732}]},"public_addr":"172.21.15.92:6825/12732","cluster_addr":"172.21.15.92:6827/12732","heartbeat_back_addr":"172.21.15.92:6831/12732","heartbeat_front_addr":"172.21.15.92:6829/12732","state":["exists","up"]},{"osd":3,"uuid":"2b9e1ff5-7b30-4565-bc03-527cf9cee5dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6809","nonce":12742},{"type":"v1","addr":"172.21.15.92:6812","nonce":12742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":12742},{"type":"v1","addr":"172.21.15.92:6816","nonce":12742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":12742},{"type":"v1","addr":"172.21.15.92:6823","nonce":12742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":12742},{"type":"v1","addr":"172.21.15.92:6819","nonce":12742}]},"public_addr":"172.21.15.92:6812/12742","cluster_addr":"172.21.15.92:6816/12742","heartbeat_back_addr":"172.21.15.92:6823/12742","heartbeat_front_addr":"172.21.15.92:6819/12742","state":["exists","up"]},{"osd":4,"uuid":"e75baa38-b785-4bde-9681-fe474fbc202d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6817","nonce":12991},{"type":"v1","addr":"172.21.15.13:6819","nonce":12991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6821","nonce":12991},{"type":"v1","addr":"172.21.15.13:6823","nonce":12991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6830","nonce":12991},{"type":"v1","addr":"172.21.15.13:6831","nonce":12991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6825","nonce":12991},{"type":"v1","addr":"172.21.15.13:6828","nonce":12991}]},"public_addr":"172.21.15.13:6819/12991","cluster_addr":"172.21.15.13:6823/12991","heartbeat_back_addr":"172.21.15.13:6831/12991","heartbeat_front_addr":"172.21.15.13:6828/12991","state":["exists","up"]},{"osd":5,"uuid":"a45307aa-ecba-42fc-9a72-5b2e4b1e3117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6800","nonce":12990},{"type":"v1","addr":"172.21.15.13:6801","nonce":12990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6802","nonce":12990},{"type":"v1","addr":"172.21.15.13:6803","nonce":12990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6806","nonce":12990},{"type":"v1","addr":"172.21.15.13:6807","nonce":12990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6804","nonce":12990},{"type":"v1","addr":"172.21.15.13:6805","nonce":12990}]},"public_addr":"172.21.15.13:6801/12990","cluster_addr":"172.21.15.13:6803/12990","heartbeat_back_addr":"172.21.15.13:6807/12990","heartbeat_front_addr":"172.21.15.13:6805/12990","state":["exists","up"]},{"osd":6,"uuid":"9c6a25e5-0753-4079-9873-09a212a61f51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6808","nonce":13001},{"type":"v1","addr":"172.21.15.13:6809","nonce":13001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6810","nonce":13001},{"type":"v1","addr":"172.21.15.13:6811","nonce":13001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6814","nonce":13001},{"type":"v1","addr":"172.21.15.13:6815","nonce":13001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6812","nonce":13001},{"type":"v1","addr":"172.21.15.13:6813","nonce":13001}]},"public_addr":"172.21.15.13:6809/13001","cluster_addr":"172.21.15.13:6811/13001","heartbeat_back_addr":"172.21.15.13:6815/13001","heartbeat_front_addr":"172.21.15.13:6813/13001","state":["exists","up"]},{"osd":7,"uuid":"2cb56fe5-4a39-4599-8f46-0d452347ee33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6816","nonce":12992},{"type":"v1","addr":"172.21.15.13:6818","nonce":12992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6820","nonce":12992},{"type":"v1","addr":"172.21.15.13:6822","nonce":12992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6827","nonce":12992},{"type":"v1","addr":"172.21.15.13:6829","nonce":12992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6824","nonce":12992},{"type":"v1","addr":"172.21.15.13:6826","nonce":12992}]},"public_addr":"172.21.15.13:6818/12992","cluster_addr":"172.21.15.13:6822/12992","heartbeat_back_addr":"172.21.15.13:6829/12992","heartbeat_front_addr":"172.21.15.13:6826/12992","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:36.753058+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.289494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.306290+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.329765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.282180+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:36.824431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.102505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.281355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T20:23:42.377 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-11-16T20:23:42.378 INFO:tasks.ceph:Creating RBD pool 2020-11-16T20:23:42.378 INFO:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-16T20:23:43.483 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2020-11-16T20:23:43.497 INFO:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-16T20:23:44.492 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-16T20:23:44.507 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-16T20:23:44.508 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T20:23:44.508 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-16T20:23:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:44.804 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"fsid":"3fc8a474-4fd1-4d15-a656-0a82bbfce204","created":"2020-11-16T20:23:31.699731+0000","modified":"2020-11-16T20:23:44.470334+0000","last_up_change":"2020-11-16T20:23:38.624506+0000","last_in_change":"2020-11-16T20:23:38.624506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T20:23:35.658847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T20:23:42.733993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"041f4de8-6113-42d0-89ef-3ff955e898a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":12741},{"type":"v1","addr":"172.21.15.92:6801","nonce":12741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":12741},{"type":"v1","addr":"172.21.15.92:6803","nonce":12741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":12741},{"type":"v1","addr":"172.21.15.92:6807","nonce":12741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":12741},{"type":"v1","addr":"172.21.15.92:6805","nonce":12741}]},"public_addr":"172.21.15.92:6801/12741","cluster_addr":"172.21.15.92:6803/12741","heartbeat_back_addr":"172.21.15.92:6807/12741","heartbeat_front_addr":"172.21.15.92:6805/12741","state":["exists","up"]},{"osd":1,"uuid":"cf43191f-7015-40ca-a363-9892eed40e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":12731},{"type":"v1","addr":"172.21.15.92:6810","nonce":12731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":12731},{"type":"v1","addr":"172.21.15.92:6813","nonce":12731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":12731},{"type":"v1","addr":"172.21.15.92:6822","nonce":12731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":12731},{"type":"v1","addr":"172.21.15.92:6817","nonce":12731}]},"public_addr":"172.21.15.92:6810/12731","cluster_addr":"172.21.15.92:6813/12731","heartbeat_back_addr":"172.21.15.92:6822/12731","heartbeat_front_addr":"172.21.15.92:6817/12731","state":["exists","up"]},{"osd":2,"uuid":"e42e4433-f9d2-4e5d-9cff-2a1ef11c6736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":12732},{"type":"v1","addr":"172.21.15.92:6825","nonce":12732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":12732},{"type":"v1","addr":"172.21.15.92:6827","nonce":12732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":12732},{"type":"v1","addr":"172.21.15.92:6831","nonce":12732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":12732},{"type":"v1","addr":"172.21.15.92:6829","nonce":12732}]},"public_addr":"172.21.15.92:6825/12732","cluster_addr":"172.21.15.92:6827/12732","heartbeat_back_addr":"172.21.15.92:6831/12732","heartbeat_front_addr":"172.21.15.92:6829/12732","state":["exists","up"]},{"osd":3,"uuid":"2b9e1ff5-7b30-4565-bc03-527cf9cee5dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6809","nonce":12742},{"type":"v1","addr":"172.21.15.92:6812","nonce":12742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":12742},{"type":"v1","addr":"172.21.15.92:6816","nonce":12742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":12742},{"type":"v1","addr":"172.21.15.92:6823","nonce":12742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":12742},{"type":"v1","addr":"172.21.15.92:6819","nonce":12742}]},"public_addr":"172.21.15.92:6812/12742","cluster_addr":"172.21.15.92:6816/12742","heartbeat_back_addr":"172.21.15.92:6823/12742","heartbeat_front_addr":"172.21.15.92:6819/12742","state":["exists","up"]},{"osd":4,"uuid":"e75baa38-b785-4bde-9681-fe474fbc202d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6817","nonce":12991},{"type":"v1","addr":"172.21.15.13:6819","nonce":12991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6821","nonce":12991},{"type":"v1","addr":"172.21.15.13:6823","nonce":12991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6830","nonce":12991},{"type":"v1","addr":"172.21.15.13:6831","nonce":12991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6825","nonce":12991},{"type":"v1","addr":"172.21.15.13:6828","nonce":12991}]},"public_addr":"172.21.15.13:6819/12991","cluster_addr":"172.21.15.13:6823/12991","heartbeat_back_addr":"172.21.15.13:6831/12991","heartbeat_front_addr":"172.21.15.13:6828/12991","state":["exists","up"]},{"osd":5,"uuid":"a45307aa-ecba-42fc-9a72-5b2e4b1e3117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6800","nonce":12990},{"type":"v1","addr":"172.21.15.13:6801","nonce":12990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6802","nonce":12990},{"type":"v1","addr":"172.21.15.13:6803","nonce":12990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6806","nonce":12990},{"type":"v1","addr":"172.21.15.13:6807","nonce":12990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6804","nonce":12990},{"type":"v1","addr":"172.21.15.13:6805","nonce":12990}]},"public_addr":"172.21.15.13:6801/12990","cluster_addr":"172.21.15.13:6803/12990","heartbeat_back_addr":"172.21.15.13:6807/12990","heartbeat_front_addr":"172.21.15.13:6805/12990","state":["exists","up"]},{"osd":6,"uuid":"9c6a25e5-0753-4079-9873-09a212a61f51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6808","nonce":13001},{"type":"v1","addr":"172.21.15.13:6809","nonce":13001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6810","nonce":13001},{"type":"v1","addr":"172.21.15.13:6811","nonce":13001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6814","nonce":13001},{"type":"v1","addr":"172.21.15.13:6815","nonce":13001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6812","nonce":13001},{"type":"v1","addr":"172.21.15.13:6813","nonce":13001}]},"public_addr":"172.21.15.13:6809/13001","cluster_addr":"172.21.15.13:6811/13001","heartbeat_back_addr":"172.21.15.13:6815/13001","heartbeat_front_addr":"172.21.15.13:6813/13001","state":["exists","up"]},{"osd":7,"uuid":"2cb56fe5-4a39-4599-8f46-0d452347ee33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6816","nonce":12992},{"type":"v1","addr":"172.21.15.13:6818","nonce":12992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6820","nonce":12992},{"type":"v1","addr":"172.21.15.13:6822","nonce":12992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6827","nonce":12992},{"type":"v1","addr":"172.21.15.13:6829","nonce":12992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6824","nonce":12992},{"type":"v1","addr":"172.21.15.13:6826","nonce":12992}]},"public_addr":"172.21.15.13:6818/12992","cluster_addr":"172.21.15.13:6822/12992","heartbeat_back_addr":"172.21.15.13:6829/12992","heartbeat_front_addr":"172.21.15.13:6826/12992","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:36.753058+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.289494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.306290+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.329765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.282180+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:36.824431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.102505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.281355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T20:23:44.819 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-16T20:23:35.658847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-16T20:23:42.733993+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-16T20:23:44.819 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-16T20:23:45.132 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2020-11-16T20:23:45.147 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-16T20:23:45.480 INFO:teuthology.orchestra.run.smithi092.stderr:2020-11-16T20:23:45.487+0000 7fd6ab7fe700 0 --2- 172.21.15.92:0/801917198 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd6ac0a9890 0x7fd6ac0a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2020-11-16T20:23:45.648 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2020-11-16T20:23:45.662 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-16T20:23:45.662 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-16T20:23:45.663 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2020-11-16T20:23:45.995 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:46.011 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"active_gid":4111,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":12577},{"type":"v1","addr":"172.21.15.92:6833","nonce":12577}]},"active_addr":"172.21.15.92:6833/12577","active_change":"2020-11-16T20:23:35.597516+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":4104,"name":"x","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2647985223}]},{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1915072835}]},{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":382069393}]}]}} 2020-11-16T20:23:46.015 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-16T20:23:46.016 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-16T20:23:46.016 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T20:23:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:46.318 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"fsid":"3fc8a474-4fd1-4d15-a656-0a82bbfce204","created":"2020-11-16T20:23:31.699731+0000","modified":"2020-11-16T20:23:45.479350+0000","last_up_change":"2020-11-16T20:23:38.624506+0000","last_in_change":"2020-11-16T20:23:38.624506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T20:23:35.658847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T20:23:42.733993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"041f4de8-6113-42d0-89ef-3ff955e898a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":12741},{"type":"v1","addr":"172.21.15.92:6801","nonce":12741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":12741},{"type":"v1","addr":"172.21.15.92:6803","nonce":12741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":12741},{"type":"v1","addr":"172.21.15.92:6807","nonce":12741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":12741},{"type":"v1","addr":"172.21.15.92:6805","nonce":12741}]},"public_addr":"172.21.15.92:6801/12741","cluster_addr":"172.21.15.92:6803/12741","heartbeat_back_addr":"172.21.15.92:6807/12741","heartbeat_front_addr":"172.21.15.92:6805/12741","state":["exists","up"]},{"osd":1,"uuid":"cf43191f-7015-40ca-a363-9892eed40e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":12731},{"type":"v1","addr":"172.21.15.92:6810","nonce":12731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":12731},{"type":"v1","addr":"172.21.15.92:6813","nonce":12731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":12731},{"type":"v1","addr":"172.21.15.92:6822","nonce":12731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":12731},{"type":"v1","addr":"172.21.15.92:6817","nonce":12731}]},"public_addr":"172.21.15.92:6810/12731","cluster_addr":"172.21.15.92:6813/12731","heartbeat_back_addr":"172.21.15.92:6822/12731","heartbeat_front_addr":"172.21.15.92:6817/12731","state":["exists","up"]},{"osd":2,"uuid":"e42e4433-f9d2-4e5d-9cff-2a1ef11c6736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":12732},{"type":"v1","addr":"172.21.15.92:6825","nonce":12732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":12732},{"type":"v1","addr":"172.21.15.92:6827","nonce":12732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":12732},{"type":"v1","addr":"172.21.15.92:6831","nonce":12732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":12732},{"type":"v1","addr":"172.21.15.92:6829","nonce":12732}]},"public_addr":"172.21.15.92:6825/12732","cluster_addr":"172.21.15.92:6827/12732","heartbeat_back_addr":"172.21.15.92:6831/12732","heartbeat_front_addr":"172.21.15.92:6829/12732","state":["exists","up"]},{"osd":3,"uuid":"2b9e1ff5-7b30-4565-bc03-527cf9cee5dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6809","nonce":12742},{"type":"v1","addr":"172.21.15.92:6812","nonce":12742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":12742},{"type":"v1","addr":"172.21.15.92:6816","nonce":12742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":12742},{"type":"v1","addr":"172.21.15.92:6823","nonce":12742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":12742},{"type":"v1","addr":"172.21.15.92:6819","nonce":12742}]},"public_addr":"172.21.15.92:6812/12742","cluster_addr":"172.21.15.92:6816/12742","heartbeat_back_addr":"172.21.15.92:6823/12742","heartbeat_front_addr":"172.21.15.92:6819/12742","state":["exists","up"]},{"osd":4,"uuid":"e75baa38-b785-4bde-9681-fe474fbc202d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6817","nonce":12991},{"type":"v1","addr":"172.21.15.13:6819","nonce":12991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6821","nonce":12991},{"type":"v1","addr":"172.21.15.13:6823","nonce":12991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6830","nonce":12991},{"type":"v1","addr":"172.21.15.13:6831","nonce":12991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6825","nonce":12991},{"type":"v1","addr":"172.21.15.13:6828","nonce":12991}]},"public_addr":"172.21.15.13:6819/12991","cluster_addr":"172.21.15.13:6823/12991","heartbeat_back_addr":"172.21.15.13:6831/12991","heartbeat_front_addr":"172.21.15.13:6828/12991","state":["exists","up"]},{"osd":5,"uuid":"a45307aa-ecba-42fc-9a72-5b2e4b1e3117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6800","nonce":12990},{"type":"v1","addr":"172.21.15.13:6801","nonce":12990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6802","nonce":12990},{"type":"v1","addr":"172.21.15.13:6803","nonce":12990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6806","nonce":12990},{"type":"v1","addr":"172.21.15.13:6807","nonce":12990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6804","nonce":12990},{"type":"v1","addr":"172.21.15.13:6805","nonce":12990}]},"public_addr":"172.21.15.13:6801/12990","cluster_addr":"172.21.15.13:6803/12990","heartbeat_back_addr":"172.21.15.13:6807/12990","heartbeat_front_addr":"172.21.15.13:6805/12990","state":["exists","up"]},{"osd":6,"uuid":"9c6a25e5-0753-4079-9873-09a212a61f51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6808","nonce":13001},{"type":"v1","addr":"172.21.15.13:6809","nonce":13001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6810","nonce":13001},{"type":"v1","addr":"172.21.15.13:6811","nonce":13001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6814","nonce":13001},{"type":"v1","addr":"172.21.15.13:6815","nonce":13001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6812","nonce":13001},{"type":"v1","addr":"172.21.15.13:6813","nonce":13001}]},"public_addr":"172.21.15.13:6809/13001","cluster_addr":"172.21.15.13:6811/13001","heartbeat_back_addr":"172.21.15.13:6815/13001","heartbeat_front_addr":"172.21.15.13:6813/13001","state":["exists","up"]},{"osd":7,"uuid":"2cb56fe5-4a39-4599-8f46-0d452347ee33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6816","nonce":12992},{"type":"v1","addr":"172.21.15.13:6818","nonce":12992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6820","nonce":12992},{"type":"v1","addr":"172.21.15.13:6822","nonce":12992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6827","nonce":12992},{"type":"v1","addr":"172.21.15.13:6829","nonce":12992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6824","nonce":12992},{"type":"v1","addr":"172.21.15.13:6826","nonce":12992}]},"public_addr":"172.21.15.13:6818/12992","cluster_addr":"172.21.15.13:6822/12992","heartbeat_back_addr":"172.21.15.13:6829/12992","heartbeat_front_addr":"172.21.15.13:6826/12992","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:36.753058+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.289494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.306290+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.329765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.282180+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:36.824431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.102505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.281355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T20:23:46.332 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-16T20:23:46.333 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-16T20:23:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:46.628 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"fsid":"3fc8a474-4fd1-4d15-a656-0a82bbfce204","created":"2020-11-16T20:23:31.699731+0000","modified":"2020-11-16T20:23:45.479350+0000","last_up_change":"2020-11-16T20:23:38.624506+0000","last_in_change":"2020-11-16T20:23:38.624506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-16T20:23:35.658847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-16T20:23:42.733993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"041f4de8-6113-42d0-89ef-3ff955e898a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":12741},{"type":"v1","addr":"172.21.15.92:6801","nonce":12741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":12741},{"type":"v1","addr":"172.21.15.92:6803","nonce":12741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":12741},{"type":"v1","addr":"172.21.15.92:6807","nonce":12741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":12741},{"type":"v1","addr":"172.21.15.92:6805","nonce":12741}]},"public_addr":"172.21.15.92:6801/12741","cluster_addr":"172.21.15.92:6803/12741","heartbeat_back_addr":"172.21.15.92:6807/12741","heartbeat_front_addr":"172.21.15.92:6805/12741","state":["exists","up"]},{"osd":1,"uuid":"cf43191f-7015-40ca-a363-9892eed40e46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":12731},{"type":"v1","addr":"172.21.15.92:6810","nonce":12731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":12731},{"type":"v1","addr":"172.21.15.92:6813","nonce":12731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":12731},{"type":"v1","addr":"172.21.15.92:6822","nonce":12731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":12731},{"type":"v1","addr":"172.21.15.92:6817","nonce":12731}]},"public_addr":"172.21.15.92:6810/12731","cluster_addr":"172.21.15.92:6813/12731","heartbeat_back_addr":"172.21.15.92:6822/12731","heartbeat_front_addr":"172.21.15.92:6817/12731","state":["exists","up"]},{"osd":2,"uuid":"e42e4433-f9d2-4e5d-9cff-2a1ef11c6736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":12732},{"type":"v1","addr":"172.21.15.92:6825","nonce":12732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":12732},{"type":"v1","addr":"172.21.15.92:6827","nonce":12732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":12732},{"type":"v1","addr":"172.21.15.92:6831","nonce":12732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":12732},{"type":"v1","addr":"172.21.15.92:6829","nonce":12732}]},"public_addr":"172.21.15.92:6825/12732","cluster_addr":"172.21.15.92:6827/12732","heartbeat_back_addr":"172.21.15.92:6831/12732","heartbeat_front_addr":"172.21.15.92:6829/12732","state":["exists","up"]},{"osd":3,"uuid":"2b9e1ff5-7b30-4565-bc03-527cf9cee5dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6809","nonce":12742},{"type":"v1","addr":"172.21.15.92:6812","nonce":12742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":12742},{"type":"v1","addr":"172.21.15.92:6816","nonce":12742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":12742},{"type":"v1","addr":"172.21.15.92:6823","nonce":12742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":12742},{"type":"v1","addr":"172.21.15.92:6819","nonce":12742}]},"public_addr":"172.21.15.92:6812/12742","cluster_addr":"172.21.15.92:6816/12742","heartbeat_back_addr":"172.21.15.92:6823/12742","heartbeat_front_addr":"172.21.15.92:6819/12742","state":["exists","up"]},{"osd":4,"uuid":"e75baa38-b785-4bde-9681-fe474fbc202d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6817","nonce":12991},{"type":"v1","addr":"172.21.15.13:6819","nonce":12991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6821","nonce":12991},{"type":"v1","addr":"172.21.15.13:6823","nonce":12991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6830","nonce":12991},{"type":"v1","addr":"172.21.15.13:6831","nonce":12991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6825","nonce":12991},{"type":"v1","addr":"172.21.15.13:6828","nonce":12991}]},"public_addr":"172.21.15.13:6819/12991","cluster_addr":"172.21.15.13:6823/12991","heartbeat_back_addr":"172.21.15.13:6831/12991","heartbeat_front_addr":"172.21.15.13:6828/12991","state":["exists","up"]},{"osd":5,"uuid":"a45307aa-ecba-42fc-9a72-5b2e4b1e3117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6800","nonce":12990},{"type":"v1","addr":"172.21.15.13:6801","nonce":12990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6802","nonce":12990},{"type":"v1","addr":"172.21.15.13:6803","nonce":12990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6806","nonce":12990},{"type":"v1","addr":"172.21.15.13:6807","nonce":12990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6804","nonce":12990},{"type":"v1","addr":"172.21.15.13:6805","nonce":12990}]},"public_addr":"172.21.15.13:6801/12990","cluster_addr":"172.21.15.13:6803/12990","heartbeat_back_addr":"172.21.15.13:6807/12990","heartbeat_front_addr":"172.21.15.13:6805/12990","state":["exists","up"]},{"osd":6,"uuid":"9c6a25e5-0753-4079-9873-09a212a61f51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6808","nonce":13001},{"type":"v1","addr":"172.21.15.13:6809","nonce":13001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6810","nonce":13001},{"type":"v1","addr":"172.21.15.13:6811","nonce":13001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6814","nonce":13001},{"type":"v1","addr":"172.21.15.13:6815","nonce":13001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6812","nonce":13001},{"type":"v1","addr":"172.21.15.13:6813","nonce":13001}]},"public_addr":"172.21.15.13:6809/13001","cluster_addr":"172.21.15.13:6811/13001","heartbeat_back_addr":"172.21.15.13:6815/13001","heartbeat_front_addr":"172.21.15.13:6813/13001","state":["exists","up"]},{"osd":7,"uuid":"2cb56fe5-4a39-4599-8f46-0d452347ee33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6816","nonce":12992},{"type":"v1","addr":"172.21.15.13:6818","nonce":12992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6820","nonce":12992},{"type":"v1","addr":"172.21.15.13:6822","nonce":12992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6827","nonce":12992},{"type":"v1","addr":"172.21.15.13:6829","nonce":12992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.13:6824","nonce":12992},{"type":"v1","addr":"172.21.15.13:6826","nonce":12992}]},"public_addr":"172.21.15.13:6818/12992","cluster_addr":"172.21.15.13:6822/12992","heartbeat_back_addr":"172.21.15.13:6829/12992","heartbeat_front_addr":"172.21.15.13:6826/12992","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:36.753058+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.289494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.306290+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.329765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.282180+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:36.824431+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.102505+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-16T20:23:37.281355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-16T20:23:50.787 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2020-11-16T20:23:50.976 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:50.976 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2020-11-16T20:23:51.159 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:51.159 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2020-11-16T20:23:51.350 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:51.350 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2020-11-16T20:23:51.539 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:51.540 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2020-11-16T20:23:51.728 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:51.729 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2020-11-16T20:23:51.920 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:51.921 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2020-11-16T20:23:52.104 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:52.104 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2020-11-16T20:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:52.289 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-16T20:23:52.603 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:52.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2020-11-16T20:23:52.617 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2020-11-16T20:23:52.913 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:52.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2020-11-16T20:23:52.927 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-16T20:23:53.228 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:53.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2020-11-16T20:23:53.242 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2020-11-16T20:23:53.541 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:53.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2020-11-16T20:23:53.556 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2020-11-16T20:23:53.864 INFO:teuthology.orchestra.run.smithi092.stdout:55834574851 2020-11-16T20:23:53.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2020-11-16T20:23:54.884 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2020-11-16T20:23:55.193 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:55.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2020-11-16T20:23:55.207 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2020-11-16T20:23:55.502 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:55.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2020-11-16T20:23:55.517 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2020-11-16T20:23:55.804 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:55.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2020-11-16T20:23:55.819 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2020-11-16T20:23:56.110 INFO:teuthology.orchestra.run.smithi092.stdout:55834574852 2020-11-16T20:23:56.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2020-11-16T20:23:56.124 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-16T20:23:56.125 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T20:23:56.417 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:56.419 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2020-11-16T20:23:56.431 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2020-11-16T20:23:55.616838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39808,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38336,"kb_avail":754934912,"statfs":{"total":773094113280,"available":773053349888,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39256064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.124748"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.606444+0000","last_change":"2020-11-16T20:23:44.605343+0000","last_active":"2020-11-16T20:23:44.606444+0000","last_peered":"2020-11-16T20:23:44.606444+0000","last_clean":"2020-11-16T20:23:44.606444+0000","last_became_active":"2020-11-16T20:23:44.604919+0000","last_became_peered":"2020-11-16T20:23:44.604919+0000","last_unstale":"2020-11-16T20:23:44.606444+0000","last_undegraded":"2020-11-16T20:23:44.606444+0000","last_fullsized":"2020-11-16T20:23:44.606444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600239+0000","last_change":"2020-11-16T20:23:44.598654+0000","last_active":"2020-11-16T20:23:44.600239+0000","last_peered":"2020-11-16T20:23:44.600239+0000","last_clean":"2020-11-16T20:23:44.600239+0000","last_became_active":"2020-11-16T20:23:44.597867+0000","last_became_peered":"2020-11-16T20:23:44.597867+0000","last_unstale":"2020-11-16T20:23:44.600239+0000","last_undegraded":"2020-11-16T20:23:44.600239+0000","last_fullsized":"2020-11-16T20:23:44.600239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502968+0000","last_change":"2020-11-16T20:23:44.501820+0000","last_active":"2020-11-16T20:23:44.502968+0000","last_peered":"2020-11-16T20:23:44.502968+0000","last_clean":"2020-11-16T20:23:44.502968+0000","last_became_active":"2020-11-16T20:23:44.501424+0000","last_became_peered":"2020-11-16T20:23:44.501424+0000","last_unstale":"2020-11-16T20:23:44.502968+0000","last_undegraded":"2020-11-16T20:23:44.502968+0000","last_fullsized":"2020-11-16T20:23:44.502968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.504330+0000","last_change":"2020-11-16T20:23:44.503055+0000","last_active":"2020-11-16T20:23:44.504330+0000","last_peered":"2020-11-16T20:23:44.504330+0000","last_clean":"2020-11-16T20:23:44.504330+0000","last_became_active":"2020-11-16T20:23:44.502588+0000","last_became_peered":"2020-11-16T20:23:44.502588+0000","last_unstale":"2020-11-16T20:23:44.504330+0000","last_undegraded":"2020-11-16T20:23:44.504330+0000","last_fullsized":"2020-11-16T20:23:44.504330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.493541+0000","last_change":"2020-11-16T20:23:39.780867+0000","last_active":"2020-11-16T20:23:44.493541+0000","last_peered":"2020-11-16T20:23:44.493541+0000","last_clean":"2020-11-16T20:23:44.493541+0000","last_became_active":"2020-11-16T20:23:39.779329+0000","last_became_peered":"2020-11-16T20:23:39.779329+0000","last_unstale":"2020-11-16T20:23:44.493541+0000","last_undegraded":"2020-11-16T20:23:44.493541+0000","last_fullsized":"2020-11-16T20:23:44.493541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:23:36.602638+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502939+0000","last_change":"2020-11-16T20:23:44.501339+0000","last_active":"2020-11-16T20:23:44.502939+0000","last_peered":"2020-11-16T20:23:44.502939+0000","last_clean":"2020-11-16T20:23:44.502939+0000","last_became_active":"2020-11-16T20:23:44.500782+0000","last_became_peered":"2020-11-16T20:23:44.500782+0000","last_unstale":"2020-11-16T20:23:44.502939+0000","last_undegraded":"2020-11-16T20:23:44.502939+0000","last_fullsized":"2020-11-16T20:23:44.502939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.494389+0000","last_change":"2020-11-16T20:23:44.493249+0000","last_active":"2020-11-16T20:23:44.494389+0000","last_peered":"2020-11-16T20:23:44.494389+0000","last_clean":"2020-11-16T20:23:44.494389+0000","last_became_active":"2020-11-16T20:23:44.492834+0000","last_became_peered":"2020-11-16T20:23:44.492834+0000","last_unstale":"2020-11-16T20:23:44.494389+0000","last_undegraded":"2020-11-16T20:23:44.494389+0000","last_fullsized":"2020-11-16T20:23:44.494389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.607721+0000","last_change":"2020-11-16T20:23:44.606663+0000","last_active":"2020-11-16T20:23:44.607721+0000","last_peered":"2020-11-16T20:23:44.607721+0000","last_clean":"2020-11-16T20:23:44.607721+0000","last_became_active":"2020-11-16T20:23:44.605904+0000","last_became_peered":"2020-11-16T20:23:44.605904+0000","last_unstale":"2020-11-16T20:23:44.607721+0000","last_undegraded":"2020-11-16T20:23:44.607721+0000","last_fullsized":"2020-11-16T20:23:44.607721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600231+0000","last_change":"2020-11-16T20:23:44.599215+0000","last_active":"2020-11-16T20:23:44.600231+0000","last_peered":"2020-11-16T20:23:44.600231+0000","last_clean":"2020-11-16T20:23:44.600231+0000","last_became_active":"2020-11-16T20:23:44.597937+0000","last_became_peered":"2020-11-16T20:23:44.597937+0000","last_unstale":"2020-11-16T20:23:44.600231+0000","last_undegraded":"2020-11-16T20:23:44.600231+0000","last_fullsized":"2020-11-16T20:23:44.600231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4920,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366920,"statfs":{"total":96636764160,"available":96631726080,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.343}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5289999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4089999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.581}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.794}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2210000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9350000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0609999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3330000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.04}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.294}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.778}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.468}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7269999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8480000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.948}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3410000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6589999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.909}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2290000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0550000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0179999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1789999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5960000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8279999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.966}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9780000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.556}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.347}]}]}]}} 2020-11-16T20:23:56.434 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T20:23:56.733 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:56.735 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2020-11-16T20:23:56.753 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2020-11-16T20:23:55.616838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39808,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38336,"kb_avail":754934912,"statfs":{"total":773094113280,"available":773053349888,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39256064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.124748"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.606444+0000","last_change":"2020-11-16T20:23:44.605343+0000","last_active":"2020-11-16T20:23:44.606444+0000","last_peered":"2020-11-16T20:23:44.606444+0000","last_clean":"2020-11-16T20:23:44.606444+0000","last_became_active":"2020-11-16T20:23:44.604919+0000","last_became_peered":"2020-11-16T20:23:44.604919+0000","last_unstale":"2020-11-16T20:23:44.606444+0000","last_undegraded":"2020-11-16T20:23:44.606444+0000","last_fullsized":"2020-11-16T20:23:44.606444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600239+0000","last_change":"2020-11-16T20:23:44.598654+0000","last_active":"2020-11-16T20:23:44.600239+0000","last_peered":"2020-11-16T20:23:44.600239+0000","last_clean":"2020-11-16T20:23:44.600239+0000","last_became_active":"2020-11-16T20:23:44.597867+0000","last_became_peered":"2020-11-16T20:23:44.597867+0000","last_unstale":"2020-11-16T20:23:44.600239+0000","last_undegraded":"2020-11-16T20:23:44.600239+0000","last_fullsized":"2020-11-16T20:23:44.600239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502968+0000","last_change":"2020-11-16T20:23:44.501820+0000","last_active":"2020-11-16T20:23:44.502968+0000","last_peered":"2020-11-16T20:23:44.502968+0000","last_clean":"2020-11-16T20:23:44.502968+0000","last_became_active":"2020-11-16T20:23:44.501424+0000","last_became_peered":"2020-11-16T20:23:44.501424+0000","last_unstale":"2020-11-16T20:23:44.502968+0000","last_undegraded":"2020-11-16T20:23:44.502968+0000","last_fullsized":"2020-11-16T20:23:44.502968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.504330+0000","last_change":"2020-11-16T20:23:44.503055+0000","last_active":"2020-11-16T20:23:44.504330+0000","last_peered":"2020-11-16T20:23:44.504330+0000","last_clean":"2020-11-16T20:23:44.504330+0000","last_became_active":"2020-11-16T20:23:44.502588+0000","last_became_peered":"2020-11-16T20:23:44.502588+0000","last_unstale":"2020-11-16T20:23:44.504330+0000","last_undegraded":"2020-11-16T20:23:44.504330+0000","last_fullsized":"2020-11-16T20:23:44.504330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.493541+0000","last_change":"2020-11-16T20:23:39.780867+0000","last_active":"2020-11-16T20:23:44.493541+0000","last_peered":"2020-11-16T20:23:44.493541+0000","last_clean":"2020-11-16T20:23:44.493541+0000","last_became_active":"2020-11-16T20:23:39.779329+0000","last_became_peered":"2020-11-16T20:23:39.779329+0000","last_unstale":"2020-11-16T20:23:44.493541+0000","last_undegraded":"2020-11-16T20:23:44.493541+0000","last_fullsized":"2020-11-16T20:23:44.493541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:23:36.602638+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502939+0000","last_change":"2020-11-16T20:23:44.501339+0000","last_active":"2020-11-16T20:23:44.502939+0000","last_peered":"2020-11-16T20:23:44.502939+0000","last_clean":"2020-11-16T20:23:44.502939+0000","last_became_active":"2020-11-16T20:23:44.500782+0000","last_became_peered":"2020-11-16T20:23:44.500782+0000","last_unstale":"2020-11-16T20:23:44.502939+0000","last_undegraded":"2020-11-16T20:23:44.502939+0000","last_fullsized":"2020-11-16T20:23:44.502939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.494389+0000","last_change":"2020-11-16T20:23:44.493249+0000","last_active":"2020-11-16T20:23:44.494389+0000","last_peered":"2020-11-16T20:23:44.494389+0000","last_clean":"2020-11-16T20:23:44.494389+0000","last_became_active":"2020-11-16T20:23:44.492834+0000","last_became_peered":"2020-11-16T20:23:44.492834+0000","last_unstale":"2020-11-16T20:23:44.494389+0000","last_undegraded":"2020-11-16T20:23:44.494389+0000","last_fullsized":"2020-11-16T20:23:44.494389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.607721+0000","last_change":"2020-11-16T20:23:44.606663+0000","last_active":"2020-11-16T20:23:44.607721+0000","last_peered":"2020-11-16T20:23:44.607721+0000","last_clean":"2020-11-16T20:23:44.607721+0000","last_became_active":"2020-11-16T20:23:44.605904+0000","last_became_peered":"2020-11-16T20:23:44.605904+0000","last_unstale":"2020-11-16T20:23:44.607721+0000","last_undegraded":"2020-11-16T20:23:44.607721+0000","last_fullsized":"2020-11-16T20:23:44.607721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600231+0000","last_change":"2020-11-16T20:23:44.599215+0000","last_active":"2020-11-16T20:23:44.600231+0000","last_peered":"2020-11-16T20:23:44.600231+0000","last_clean":"2020-11-16T20:23:44.600231+0000","last_became_active":"2020-11-16T20:23:44.597937+0000","last_became_peered":"2020-11-16T20:23:44.597937+0000","last_unstale":"2020-11-16T20:23:44.600231+0000","last_undegraded":"2020-11-16T20:23:44.600231+0000","last_fullsized":"2020-11-16T20:23:44.600231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4920,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366920,"statfs":{"total":96636764160,"available":96631726080,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.343}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5289999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4089999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.581}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.794}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2210000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9350000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0609999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3330000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.04}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.294}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.778}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.468}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7269999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8480000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.948}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3410000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6589999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.909}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2290000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0550000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0179999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1789999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5960000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8279999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.966}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9780000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.556}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.347}]}]}]}} 2020-11-16T20:23:56.755 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-16T20:23:56.755 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-16T20:23:56.755 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-16T20:23:56.756 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2020-11-16T20:23:57.105 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:57.120 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-16T20:23:57.120 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-16T20:23:57.121 INFO:teuthology.run_tasks:Running task thrashosds... 2020-11-16T20:23:57.131 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2020-11-16T20:23:57.131 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-kefu-testing-2020-11-16-1845'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'Health detail', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'Health detail'], 'sha1': '911cc35794d871071a8bebfb0919482263b1d18b'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '911cc35794d871071a8bebfb0919482263b1d18b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-kefu-testing-2020-11-16-1845', 'sha1': '911cc35794d871071a8bebfb0919482263b1d18b'}} 2020-11-16T20:23:57.131 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2020-11-16T20:23:57.132 INFO:tasks.thrashosds:Beginning thrashosds... 2020-11-16T20:23:57.132 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-16T20:23:57.132 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T20:23:57.432 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:57.435 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2020-11-16T20:23:57.446 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2020-11-16T20:23:55.616838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39808,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38336,"kb_avail":754934912,"statfs":{"total":773094113280,"available":773053349888,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39256064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.124748"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.606444+0000","last_change":"2020-11-16T20:23:44.605343+0000","last_active":"2020-11-16T20:23:44.606444+0000","last_peered":"2020-11-16T20:23:44.606444+0000","last_clean":"2020-11-16T20:23:44.606444+0000","last_became_active":"2020-11-16T20:23:44.604919+0000","last_became_peered":"2020-11-16T20:23:44.604919+0000","last_unstale":"2020-11-16T20:23:44.606444+0000","last_undegraded":"2020-11-16T20:23:44.606444+0000","last_fullsized":"2020-11-16T20:23:44.606444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600239+0000","last_change":"2020-11-16T20:23:44.598654+0000","last_active":"2020-11-16T20:23:44.600239+0000","last_peered":"2020-11-16T20:23:44.600239+0000","last_clean":"2020-11-16T20:23:44.600239+0000","last_became_active":"2020-11-16T20:23:44.597867+0000","last_became_peered":"2020-11-16T20:23:44.597867+0000","last_unstale":"2020-11-16T20:23:44.600239+0000","last_undegraded":"2020-11-16T20:23:44.600239+0000","last_fullsized":"2020-11-16T20:23:44.600239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502968+0000","last_change":"2020-11-16T20:23:44.501820+0000","last_active":"2020-11-16T20:23:44.502968+0000","last_peered":"2020-11-16T20:23:44.502968+0000","last_clean":"2020-11-16T20:23:44.502968+0000","last_became_active":"2020-11-16T20:23:44.501424+0000","last_became_peered":"2020-11-16T20:23:44.501424+0000","last_unstale":"2020-11-16T20:23:44.502968+0000","last_undegraded":"2020-11-16T20:23:44.502968+0000","last_fullsized":"2020-11-16T20:23:44.502968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.504330+0000","last_change":"2020-11-16T20:23:44.503055+0000","last_active":"2020-11-16T20:23:44.504330+0000","last_peered":"2020-11-16T20:23:44.504330+0000","last_clean":"2020-11-16T20:23:44.504330+0000","last_became_active":"2020-11-16T20:23:44.502588+0000","last_became_peered":"2020-11-16T20:23:44.502588+0000","last_unstale":"2020-11-16T20:23:44.504330+0000","last_undegraded":"2020-11-16T20:23:44.504330+0000","last_fullsized":"2020-11-16T20:23:44.504330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.493541+0000","last_change":"2020-11-16T20:23:39.780867+0000","last_active":"2020-11-16T20:23:44.493541+0000","last_peered":"2020-11-16T20:23:44.493541+0000","last_clean":"2020-11-16T20:23:44.493541+0000","last_became_active":"2020-11-16T20:23:39.779329+0000","last_became_peered":"2020-11-16T20:23:39.779329+0000","last_unstale":"2020-11-16T20:23:44.493541+0000","last_undegraded":"2020-11-16T20:23:44.493541+0000","last_fullsized":"2020-11-16T20:23:44.493541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:23:36.602638+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502939+0000","last_change":"2020-11-16T20:23:44.501339+0000","last_active":"2020-11-16T20:23:44.502939+0000","last_peered":"2020-11-16T20:23:44.502939+0000","last_clean":"2020-11-16T20:23:44.502939+0000","last_became_active":"2020-11-16T20:23:44.500782+0000","last_became_peered":"2020-11-16T20:23:44.500782+0000","last_unstale":"2020-11-16T20:23:44.502939+0000","last_undegraded":"2020-11-16T20:23:44.502939+0000","last_fullsized":"2020-11-16T20:23:44.502939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.494389+0000","last_change":"2020-11-16T20:23:44.493249+0000","last_active":"2020-11-16T20:23:44.494389+0000","last_peered":"2020-11-16T20:23:44.494389+0000","last_clean":"2020-11-16T20:23:44.494389+0000","last_became_active":"2020-11-16T20:23:44.492834+0000","last_became_peered":"2020-11-16T20:23:44.492834+0000","last_unstale":"2020-11-16T20:23:44.494389+0000","last_undegraded":"2020-11-16T20:23:44.494389+0000","last_fullsized":"2020-11-16T20:23:44.494389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.607721+0000","last_change":"2020-11-16T20:23:44.606663+0000","last_active":"2020-11-16T20:23:44.607721+0000","last_peered":"2020-11-16T20:23:44.607721+0000","last_clean":"2020-11-16T20:23:44.607721+0000","last_became_active":"2020-11-16T20:23:44.605904+0000","last_became_peered":"2020-11-16T20:23:44.605904+0000","last_unstale":"2020-11-16T20:23:44.607721+0000","last_undegraded":"2020-11-16T20:23:44.607721+0000","last_fullsized":"2020-11-16T20:23:44.607721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600231+0000","last_change":"2020-11-16T20:23:44.599215+0000","last_active":"2020-11-16T20:23:44.600231+0000","last_peered":"2020-11-16T20:23:44.600231+0000","last_clean":"2020-11-16T20:23:44.600231+0000","last_became_active":"2020-11-16T20:23:44.597937+0000","last_became_peered":"2020-11-16T20:23:44.597937+0000","last_unstale":"2020-11-16T20:23:44.600231+0000","last_undegraded":"2020-11-16T20:23:44.600231+0000","last_fullsized":"2020-11-16T20:23:44.600231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4920,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366920,"statfs":{"total":96636764160,"available":96631726080,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.343}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5289999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4089999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.581}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.794}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2210000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9350000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0609999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3330000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.04}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.294}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.778}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.468}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7269999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8480000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.948}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3410000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6589999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.909}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2290000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0550000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0179999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1789999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5960000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8279999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.966}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9780000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.556}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.347}]}]}]}} 2020-11-16T20:23:57.447 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T20:23:57.743 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:23:57.746 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2020-11-16T20:23:57.758 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2020-11-16T20:23:57.617402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39808,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38336,"kb_avail":754934912,"statfs":{"total":773094113280,"available":773053349888,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39256064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003352"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.606444+0000","last_change":"2020-11-16T20:23:44.605343+0000","last_active":"2020-11-16T20:23:44.606444+0000","last_peered":"2020-11-16T20:23:44.606444+0000","last_clean":"2020-11-16T20:23:44.606444+0000","last_became_active":"2020-11-16T20:23:44.604919+0000","last_became_peered":"2020-11-16T20:23:44.604919+0000","last_unstale":"2020-11-16T20:23:44.606444+0000","last_undegraded":"2020-11-16T20:23:44.606444+0000","last_fullsized":"2020-11-16T20:23:44.606444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600239+0000","last_change":"2020-11-16T20:23:44.598654+0000","last_active":"2020-11-16T20:23:44.600239+0000","last_peered":"2020-11-16T20:23:44.600239+0000","last_clean":"2020-11-16T20:23:44.600239+0000","last_became_active":"2020-11-16T20:23:44.597867+0000","last_became_peered":"2020-11-16T20:23:44.597867+0000","last_unstale":"2020-11-16T20:23:44.600239+0000","last_undegraded":"2020-11-16T20:23:44.600239+0000","last_fullsized":"2020-11-16T20:23:44.600239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502968+0000","last_change":"2020-11-16T20:23:44.501820+0000","last_active":"2020-11-16T20:23:44.502968+0000","last_peered":"2020-11-16T20:23:44.502968+0000","last_clean":"2020-11-16T20:23:44.502968+0000","last_became_active":"2020-11-16T20:23:44.501424+0000","last_became_peered":"2020-11-16T20:23:44.501424+0000","last_unstale":"2020-11-16T20:23:44.502968+0000","last_undegraded":"2020-11-16T20:23:44.502968+0000","last_fullsized":"2020-11-16T20:23:44.502968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.504330+0000","last_change":"2020-11-16T20:23:44.503055+0000","last_active":"2020-11-16T20:23:44.504330+0000","last_peered":"2020-11-16T20:23:44.504330+0000","last_clean":"2020-11-16T20:23:44.504330+0000","last_became_active":"2020-11-16T20:23:44.502588+0000","last_became_peered":"2020-11-16T20:23:44.502588+0000","last_unstale":"2020-11-16T20:23:44.504330+0000","last_undegraded":"2020-11-16T20:23:44.504330+0000","last_fullsized":"2020-11-16T20:23:44.504330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.493541+0000","last_change":"2020-11-16T20:23:39.780867+0000","last_active":"2020-11-16T20:23:44.493541+0000","last_peered":"2020-11-16T20:23:44.493541+0000","last_clean":"2020-11-16T20:23:44.493541+0000","last_became_active":"2020-11-16T20:23:39.779329+0000","last_became_peered":"2020-11-16T20:23:39.779329+0000","last_unstale":"2020-11-16T20:23:44.493541+0000","last_undegraded":"2020-11-16T20:23:44.493541+0000","last_fullsized":"2020-11-16T20:23:44.493541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:23:36.602638+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502939+0000","last_change":"2020-11-16T20:23:44.501339+0000","last_active":"2020-11-16T20:23:44.502939+0000","last_peered":"2020-11-16T20:23:44.502939+0000","last_clean":"2020-11-16T20:23:44.502939+0000","last_became_active":"2020-11-16T20:23:44.500782+0000","last_became_peered":"2020-11-16T20:23:44.500782+0000","last_unstale":"2020-11-16T20:23:44.502939+0000","last_undegraded":"2020-11-16T20:23:44.502939+0000","last_fullsized":"2020-11-16T20:23:44.502939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.494389+0000","last_change":"2020-11-16T20:23:44.493249+0000","last_active":"2020-11-16T20:23:44.494389+0000","last_peered":"2020-11-16T20:23:44.494389+0000","last_clean":"2020-11-16T20:23:44.494389+0000","last_became_active":"2020-11-16T20:23:44.492834+0000","last_became_peered":"2020-11-16T20:23:44.492834+0000","last_unstale":"2020-11-16T20:23:44.494389+0000","last_undegraded":"2020-11-16T20:23:44.494389+0000","last_fullsized":"2020-11-16T20:23:44.494389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.607721+0000","last_change":"2020-11-16T20:23:44.606663+0000","last_active":"2020-11-16T20:23:44.607721+0000","last_peered":"2020-11-16T20:23:44.607721+0000","last_clean":"2020-11-16T20:23:44.607721+0000","last_became_active":"2020-11-16T20:23:44.605904+0000","last_became_peered":"2020-11-16T20:23:44.605904+0000","last_unstale":"2020-11-16T20:23:44.607721+0000","last_undegraded":"2020-11-16T20:23:44.607721+0000","last_fullsized":"2020-11-16T20:23:44.607721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600231+0000","last_change":"2020-11-16T20:23:44.599215+0000","last_active":"2020-11-16T20:23:44.600231+0000","last_peered":"2020-11-16T20:23:44.600231+0000","last_clean":"2020-11-16T20:23:44.600231+0000","last_became_active":"2020-11-16T20:23:44.597937+0000","last_became_peered":"2020-11-16T20:23:44.597937+0000","last_unstale":"2020-11-16T20:23:44.600231+0000","last_undegraded":"2020-11-16T20:23:44.600231+0000","last_fullsized":"2020-11-16T20:23:44.600231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4920,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366920,"statfs":{"total":96636764160,"available":96631726080,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.343}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5289999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4089999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.581}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.794}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2210000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9350000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0609999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3330000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.04}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.294}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.778}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.468}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7269999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8480000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.948}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3410000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6589999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.909}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2290000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0550000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0179999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1789999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.855}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5960000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8279999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.966}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9780000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.556}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.347}]}]}]}} 2020-11-16T20:23:57.760 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-16T20:23:57.760 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2020-11-16T20:23:58.053 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 18 2020-11-16T20:23:58.054 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 3fc8a474-4fd1-4d15-a656-0a82bbfce204 2020-11-16T20:23:58.054 INFO:teuthology.orchestra.run.smithi092.stdout:created 2020-11-16T20:23:31.699731+0000 2020-11-16T20:23:58.054 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2020-11-16T20:23:45.479350+0000 2020-11-16T20:23:58.054 INFO:teuthology.orchestra.run.smithi092.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2020-11-16T20:23:58.054 INFO:teuthology.orchestra.run.smithi092.stdout:crush_version 4 2020-11-16T20:23:58.054 INFO:teuthology.orchestra.run.smithi092.stdout:full_ratio 0.95 2020-11-16T20:23:58.054 INFO:teuthology.orchestra.run.smithi092.stdout:backfillfull_ratio 0.9 2020-11-16T20:23:58.055 INFO:teuthology.orchestra.run.smithi092.stdout:nearfull_ratio 0.85 2020-11-16T20:23:58.055 INFO:teuthology.orchestra.run.smithi092.stdout:require_min_compat_client luminous 2020-11-16T20:23:58.055 INFO:teuthology.orchestra.run.smithi092.stdout:min_compat_client jewel 2020-11-16T20:23:58.055 INFO:teuthology.orchestra.run.smithi092.stdout:require_osd_release pacific 2020-11-16T20:23:58.055 INFO:teuthology.orchestra.run.smithi092.stdout:stretch_mode_enabled false 2020-11-16T20:23:58.055 INFO:teuthology.orchestra.run.smithi092.stdout:pool 1 'device_health_metrics' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 pg_num_min 1 application mgr_devicehealth 2020-11-16T20:23:58.055 INFO:teuthology.orchestra.run.smithi092.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 application rbd 2020-11-16T20:23:58.056 INFO:teuthology.orchestra.run.smithi092.stdout:max_osd 8 2020-11-16T20:23:58.056 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/12741,v1:172.21.15.92:6801/12741] [v2:172.21.15.92:6802/12741,v1:172.21.15.92:6803/12741] exists,up 041f4de8-6113-42d0-89ef-3ff955e898a6 2020-11-16T20:23:58.056 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/12731,v1:172.21.15.92:6810/12731] [v2:172.21.15.92:6811/12731,v1:172.21.15.92:6813/12731] exists,up cf43191f-7015-40ca-a363-9892eed40e46 2020-11-16T20:23:58.056 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/12732,v1:172.21.15.92:6825/12732] [v2:172.21.15.92:6826/12732,v1:172.21.15.92:6827/12732] exists,up e42e4433-f9d2-4e5d-9cff-2a1ef11c6736 2020-11-16T20:23:58.056 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6809/12742,v1:172.21.15.92:6812/12742] [v2:172.21.15.92:6814/12742,v1:172.21.15.92:6816/12742] exists,up 2b9e1ff5-7b30-4565-bc03-527cf9cee5dd 2020-11-16T20:23:58.056 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.13:6817/12991,v1:172.21.15.13:6819/12991] [v2:172.21.15.13:6821/12991,v1:172.21.15.13:6823/12991] exists,up e75baa38-b785-4bde-9681-fe474fbc202d 2020-11-16T20:23:58.056 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.13:6800/12990,v1:172.21.15.13:6801/12990] [v2:172.21.15.13:6802/12990,v1:172.21.15.13:6803/12990] exists,up a45307aa-ecba-42fc-9a72-5b2e4b1e3117 2020-11-16T20:23:58.057 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.13:6808/13001,v1:172.21.15.13:6809/13001] [v2:172.21.15.13:6810/13001,v1:172.21.15.13:6811/13001] exists,up 9c6a25e5-0753-4079-9873-09a212a61f51 2020-11-16T20:23:58.057 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.13:6816/12992,v1:172.21.15.13:6818/12992] [v2:172.21.15.13:6820/12992,v1:172.21.15.13:6822/12992] exists,up 2cb56fe5-4a39-4599-8f46-0d452347ee33 2020-11-16T20:23:58.068 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/12741,v1:172.21.15.92:6801/12741] [v2:172.21.15.92:6802/12741,v1:172.21.15.92:6803/12741] exists,up 041f4de8-6113-42d0-89ef-3ff955e898a6', 'osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/12731,v1:172.21.15.92:6810/12731] [v2:172.21.15.92:6811/12731,v1:172.21.15.92:6813/12731] exists,up cf43191f-7015-40ca-a363-9892eed40e46', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/12732,v1:172.21.15.92:6825/12732] [v2:172.21.15.92:6826/12732,v1:172.21.15.92:6827/12732] exists,up e42e4433-f9d2-4e5d-9cff-2a1ef11c6736', 'osd.3 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6809/12742,v1:172.21.15.92:6812/12742] [v2:172.21.15.92:6814/12742,v1:172.21.15.92:6816/12742] exists,up 2b9e1ff5-7b30-4565-bc03-527cf9cee5dd', 'osd.4 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.13:6817/12991,v1:172.21.15.13:6819/12991] [v2:172.21.15.13:6821/12991,v1:172.21.15.13:6823/12991] exists,up e75baa38-b785-4bde-9681-fe474fbc202d', 'osd.5 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.13:6800/12990,v1:172.21.15.13:6801/12990] [v2:172.21.15.13:6802/12990,v1:172.21.15.13:6803/12990] exists,up a45307aa-ecba-42fc-9a72-5b2e4b1e3117', 'osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.13:6808/13001,v1:172.21.15.13:6809/13001] [v2:172.21.15.13:6810/13001,v1:172.21.15.13:6811/13001] exists,up 9c6a25e5-0753-4079-9873-09a212a61f51', 'osd.7 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.13:6816/12992,v1:172.21.15.13:6818/12992] [v2:172.21.15.13:6820/12992,v1:172.21.15.13:6822/12992] exists,up 2cb56fe5-4a39-4599-8f46-0d452347ee33'] 2020-11-16T20:23:58.068 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2020-11-16T20:23:58.184 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:23:58.184 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mon.a", 2020-11-16T20:23:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster": "ceph", 2020-11-16T20:23:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2020-11-16T20:23:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "admin_socket_mode": "", 2020-11-16T20:23:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_client_required": "cephx, none", 2020-11-16T20:23:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_cluster_required": "cephx", 2020-11-16T20:23:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_debug": "true", 2020-11-16T20:23:58.185 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_mon_ticket_ttl": "43200.000000", 2020-11-16T20:23:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_service_required": "cephx", 2020-11-16T20:23:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_service_ticket_ttl": "3600.000000", 2020-11-16T20:23:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_supported": "cephx", 2020-11-16T20:23:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio": "true", 2020-11-16T20:23:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_max_queue_depth": "1024", 2020-11-16T20:23:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_poll_ms": "250", 2020-11-16T20:23:58.186 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_reap_max": "16", 2020-11-16T20:23:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_async_discard": "false", 2020-11-16T20:23:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_block_size": "4096", 2020-11-16T20:23:58.187 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio": "false", 2020-11-16T20:24:02.362 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio_log_age": "5.000000", 2020-11-16T20:24:02.362 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2020-11-16T20:24:02.363 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_inflight_ios": "false", 2020-11-16T20:24:02.363 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_enable_discard": "false", 2020-11-16T20:24:02.363 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_flock_retry": "3", 2020-11-16T20:24:02.363 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_flock_retry_interval": "0.100000", 2020-11-16T20:24:02.364 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_inject_crash": "0", 2020-11-16T20:24:02.364 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_inject_crash_flush_delay": "2", 2020-11-16T20:24:02.364 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring": "false", 2020-11-16T20:24:02.365 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_nvme_unbind_from_kernel": "false", 2020-11-16T20:24:02.365 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_type": "", 2020-11-16T20:24:02.365 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_alloc_size": "1048576", 2020-11-16T20:24:02.365 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_allocator": "hybrid", 2020-11-16T20:24:02.366 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_buffered_io": "false", 2020-11-16T20:24:02.366 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_compact_log_sync": "false", 2020-11-16T20:24:02.366 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2020-11-16T20:24:02.367 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_compact_min_size": "16777216", 2020-11-16T20:24:02.367 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_replay_check_allocations": "true", 2020-11-16T20:24:02.367 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_max_log_runway": "4194304", 2020-11-16T20:24:02.367 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_max_prefetch": "1048576", 2020-11-16T20:24:02.368 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_min_flush_size": "524288", 2020-11-16T20:24:02.368 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_min_log_runway": "1048576", 2020-11-16T20:24:02.368 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_replay_recovery": "false", 2020-11-16T20:24:02.369 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_replay_recovery_disable_compact": "false", 2020-11-16T20:24:02.369 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_shared_alloc_size": "65536", 2020-11-16T20:24:02.369 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_sync_write": "false", 2020-11-16T20:24:02.369 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2020-11-16T20:24:02.370 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2020-11-16T20:24:02.370 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2020-11-16T20:24:02.370 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_allocator": "hybrid", 2020-11-16T20:24:02.370 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2020-11-16T20:24:02.371 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2020-11-16T20:24:02.371 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2020-11-16T20:24:02.371 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bitmapallocator_span_size": "1024", 2020-11-16T20:24:02.372 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blobid_prealloc": "10240", 2020-11-16T20:24:02.372 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_create": "true", 2020-11-16T20:24:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_create": "false", 2020-11-16T20:24:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_path": "", 2020-11-16T20:24:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_size": "0", 2020-11-16T20:24:02.373 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_path": "", 2020-11-16T20:24:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_preallocate_file": "false", 2020-11-16T20:24:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_size": "107374182400", 2020-11-16T20:24:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_create": "false", 2020-11-16T20:24:02.374 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_path": "", 2020-11-16T20:24:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_size": "100663296", 2020-11-16T20:24:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs": "true", 2020-11-16T20:24:02.375 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2020-11-16T20:24:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_env_mirror": "false", 2020-11-16T20:24:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_autotune": "true", 2020-11-16T20:24:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_autotune_interval": "5.000000", 2020-11-16T20:24:02.376 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_kv_ratio": "0.400000", 2020-11-16T20:24:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_ratio": "0.400000", 2020-11-16T20:24:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size": "0", 2020-11-16T20:24:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size_hdd": "1073741824", 2020-11-16T20:24:02.377 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size_ssd": "3221225472", 2020-11-16T20:24:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_trim_interval": "0.050000", 2020-11-16T20:24:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_trim_max_skip_pinned": "64", 2020-11-16T20:24:02.378 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_type": "2q", 2020-11-16T20:24:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_clone_cow": "true", 2020-11-16T20:24:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_algorithm": "snappy", 2020-11-16T20:24:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size": "0", 2020-11-16T20:24:02.379 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2020-11-16T20:24:02.380 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2020-11-16T20:24:02.380 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size": "0", 2020-11-16T20:24:02.380 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2020-11-16T20:24:02.380 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2020-11-16T20:24:02.381 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_mode": "none", 2020-11-16T20:24:02.381 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_required_ratio": "0.875000", 2020-11-16T20:24:02.381 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_csum_type": "crc32c", 2020-11-16T20:24:02.382 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_enforce_settings": "default", 2020-11-16T20:24:02.382 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_freelist": "false", 2020-11-16T20:24:02.383 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_fsck_abort": "false", 2020-11-16T20:24:02.383 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_bug21040": "false", 2020-11-16T20:24:02.383 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2020-11-16T20:24:02.383 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_read_err": "false", 2020-11-16T20:24:02.384 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_no_reuse_blocks": "false", 2020-11-16T20:24:02.384 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_omit_block_device_write": "false", 2020-11-16T20:24:02.384 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_omit_kv_commit": "false", 2020-11-16T20:24:02.384 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2020-11-16T20:24:02.385 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_prefill": "0.000000", 2020-11-16T20:24:02.385 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_prefragment_max": "1048576", 2020-11-16T20:24:02.385 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_random_read_err": "0.000000", 2020-11-16T20:24:02.385 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2020-11-16T20:24:02.386 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_small_allocations": "0", 2020-11-16T20:24:02.386 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2020-11-16T20:24:02.386 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_default_buffered_read": "true", 2020-11-16T20:24:02.387 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_default_buffered_write": "false", 2020-11-16T20:24:02.387 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops": "0", 2020-11-16T20:24:02.387 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2020-11-16T20:24:02.387 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2020-11-16T20:24:02.388 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2020-11-16T20:24:02.388 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_max_size": "1200", 2020-11-16T20:24:02.388 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_min_size": "150", 2020-11-16T20:24:02.389 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_target_size": "500", 2020-11-16T20:24:02.389 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2020-11-16T20:24:02.389 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_freelist_blocks_per_key": "128", 2020-11-16T20:24:02.389 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2020-11-16T20:24:02.390 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2020-11-16T20:24:02.390 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mkfs": "true", 2020-11-16T20:24:02.390 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2020-11-16T20:24:02.390 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mount": "false", 2020-11-16T20:24:02.391 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mount_deep": "false", 2020-11-16T20:24:02.391 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_umount": "false", 2020-11-16T20:24:02.391 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_umount_deep": "false", 2020-11-16T20:24:02.391 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_quick_fix_on_mount": "true", 2020-11-16T20:24:02.391 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_quick_fix_threads": "2", 2020-11-16T20:24:02.391 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2020-11-16T20:24:02.391 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_gc_enable_blob_threshold": "0", 2020-11-16T20:24:02.392 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_gc_enable_total_threshold": "0", 2020-11-16T20:24:02.392 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2020-11-16T20:24:02.392 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_ignore_data_csum": "false", 2020-11-16T20:24:02.392 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_kvbackend": "rocksdb", 2020-11-16T20:24:02.393 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_collection_list_age": "60.000000", 2020-11-16T20:24:02.393 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2020-11-16T20:24:02.393 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_op_age": "5.000000", 2020-11-16T20:24:02.393 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_alloc_size": "0", 2020-11-16T20:24:02.393 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size": "0", 2020-11-16T20:24:02.393 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size_hdd": "65536", 2020-11-16T20:24:02.393 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size_ssd": "65536", 2020-11-16T20:24:02.394 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_defer_interval": "3.000000", 2020-11-16T20:24:02.394 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_deferred_txc": "32", 2020-11-16T20:24:02.394 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size": "0", 2020-11-16T20:24:02.394 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size_hdd": "4096", 2020-11-16T20:24:02.394 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size_ssd": "4096", 2020-11-16T20:24:02.394 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_nid_prealloc": "1024", 2020-11-16T20:24:02.394 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size": "0", 2020-11-16T20:24:02.395 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2020-11-16T20:24:02.395 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2020-11-16T20:24:02.395 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_retry_disk_reads": "3", 2020-11-16T20:24:02.395 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_cf": "true", 2020-11-16T20:24:02.396 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_cfs": "m(3) O(3,0-13) L", 2020-11-16T20:24:02.396 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2020-11-16T20:24:02.396 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_coremask": "0x1", 2020-11-16T20:24:02.397 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_io_sleep": "5", 2020-11-16T20:24:02.397 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_max_io_completion": "0", 2020-11-16T20:24:02.397 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_mem": "512", 2020-11-16T20:24:02.397 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_sync_submit_transaction": "false", 2020-11-16T20:24:02.397 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_bytes": "67108864", 2020-11-16T20:24:02.397 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io": "0", 2020-11-16T20:24:02.397 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2020-11-16T20:24:02.398 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2020-11-16T20:24:02.398 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2020-11-16T20:24:02.399 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_trace_rate": "0.000000", 2020-11-16T20:24:02.399 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_tracing": "false", 2020-11-16T20:24:02.399 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2020-11-16T20:24:02.399 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_reserved": "0", 2020-11-16T20:24:02.399 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2020-11-16T20:24:02.399 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2020-11-16T20:24:02.400 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_legacy_statfs": "true", 2020-11-16T20:24:02.400 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2020-11-16T20:24:02.400 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2020-11-16T20:24:02.400 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_path": "/usr/sbin/cephadm", 2020-11-16T20:24:02.400 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_directory_choose_policy": "random", 2020-11-16T20:24:02.401 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2020-11-16T20:24:02.401 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_mirror_action_update_interval": "2", 2020-11-16T20:24:02.401 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2020-11-16T20:24:02.401 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_cluster_require_signatures": "false", 2020-11-16T20:24:02.401 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_cluster_require_version": "1", 2020-11-16T20:24:02.401 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_require_signatures": "false", 2020-11-16T20:24:02.401 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_require_version": "1", 2020-11-16T20:24:02.402 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_service_require_signatures": "false", 2020-11-16T20:24:02.402 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_service_require_version": "1", 2020-11-16T20:24:02.402 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_sign_messages": "true", 2020-11-16T20:24:02.402 INFO:teuthology.orchestra.run.smithi092.stdout: "chdir": "", 2020-11-16T20:24:02.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_acl_type": "", 2020-11-16T20:24:02.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_asio_thread_count": "2", 2020-11-16T20:24:02.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_cache_mid": "0.750000", 2020-11-16T20:24:02.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_cache_size": "16384", 2020-11-16T20:24:02.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_caps_release_delay": "5", 2020-11-16T20:24:02.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_check_pool_perm": "true", 2020-11-16T20:24:02.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_force_sync_read": "false", 2020-11-16T20:24:02.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_getattr_caps": "false", 2020-11-16T20:24:02.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_inject_tick_delay": "0", 2020-11-16T20:24:02.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_die_on_failed_dentry_invalidate": "true", 2020-11-16T20:24:02.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_die_on_failed_remount": "false", 2020-11-16T20:24:02.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_dirsize_rbytes": "true", 2020-11-16T20:24:02.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_force_lazyio": "false", 2020-11-16T20:24:02.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_fs": "", 2020-11-16T20:24:02.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_inject_fixed_oldest_tid": "false", 2020-11-16T20:24:02.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_inject_release_failure": "false", 2020-11-16T20:24:02.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_max_inline_size": "4096", 2020-11-16T20:24:02.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mds_namespace": "", 2020-11-16T20:24:02.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_metadata": "", 2020-11-16T20:24:02.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_gid": "-1", 2020-11-16T20:24:02.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_timeout": "300.000000", 2020-11-16T20:24:02.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_uid": "-1", 2020-11-16T20:24:02.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mountpoint": "/", 2020-11-16T20:24:02.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_notify_timeout": "10", 2020-11-16T20:24:02.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc": "true", 2020-11-16T20:24:02.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_dirty": "104857600", 2020-11-16T20:24:02.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_dirty_age": "5.000000", 2020-11-16T20:24:02.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_objects": "1000", 2020-11-16T20:24:02.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_size": "209715200", 2020-11-16T20:24:02.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_target_dirty": "8388608", 2020-11-16T20:24:02.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_permissions": "true", 2020-11-16T20:24:02.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_quota_df": "true", 2020-11-16T20:24:02.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_max_bytes": "0", 2020-11-16T20:24:02.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_max_periods": "4", 2020-11-16T20:24:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_min": "131072", 2020-11-16T20:24:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_reconnect_stale": "false", 2020-11-16T20:24:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_shutdown_timeout": "30", 2020-11-16T20:24:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_snapdir": ".snap", 2020-11-16T20:24:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_tick_interval": "1", 2020-11-16T20:24:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_trace": "", 2020-11-16T20:24:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_try_dentry_invalidate": "false", 2020-11-16T20:24:02.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_use_faked_inos": "false", 2020-11-16T20:24:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_use_random_mds": "false", 2020-11-16T20:24:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog": "false", 2020-11-16T20:24:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog_host": "127.0.0.1", 2020-11-16T20:24:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog_port": "12201", 2020-11-16T20:24:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_monitors": "default=true", 2020-11-16T20:24:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog": "false", 2020-11-16T20:24:02.409 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2020-11-16T20:24:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog_level": "info", 2020-11-16T20:24:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "-", 2020-11-16T20:24:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_network": "", 2020-11-16T20:24:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_network_interface": "", 2020-11-16T20:24:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_isal": "false", 2020-11-16T20:24:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_level": "5", 2020-11-16T20:24:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_winsize": "-15", 2020-11-16T20:24:02.411 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zstd_level": "1", 2020-11-16T20:24:02.411 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2020-11-16T20:24:02.411 INFO:teuthology.orchestra.run.smithi092.stdout: "crash_dir": "/var/lib/ceph/crash", 2020-11-16T20:24:02.411 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_obc_lru_size": "10", 2020-11-16T20:24:02.411 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_scheduler_concurrency": "0", 2020-11-16T20:24:02.411 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location": "", 2020-11-16T20:24:02.412 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location_hook": "", 2020-11-16T20:24:02.412 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location_hook_timeout": "10", 2020-11-16T20:24:02.412 INFO:teuthology.orchestra.run.smithi092.stdout: "daemonize": "false", 2020-11-16T20:24:02.412 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_allow_any_pool_priority": "false", 2020-11-16T20:24:02.412 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asok": "1/5", 2020-11-16T20:24:02.412 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asok_assert_abort": "false", 2020-11-16T20:24:02.413 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asserts_on_shutdown": "true", 2020-11-16T20:24:02.413 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_auth": "20/20", 2020-11-16T20:24:02.413 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bdev": "1/3", 2020-11-16T20:24:02.413 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bluefs": "1/5", 2020-11-16T20:24:02.414 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bluestore": "1/5", 2020-11-16T20:24:02.414 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_buffer": "0/1", 2020-11-16T20:24:02.414 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cephfs_mirror": "20/20", 2020-11-16T20:24:02.414 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_civetweb": "1/10", 2020-11-16T20:24:02.414 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_client": "0/5", 2020-11-16T20:24:02.414 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_compressor": "1/5", 2020-11-16T20:24:02.415 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_context": "0/1", 2020-11-16T20:24:02.415 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crush": "1/1", 2020-11-16T20:24:02.415 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crypto": "1/5", 2020-11-16T20:24:02.415 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_deliberately_leak_memory": "false", 2020-11-16T20:24:02.415 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_disable_randomized_ping": "false", 2020-11-16T20:24:02.415 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_dpdk": "1/5", 2020-11-16T20:24:02.415 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_eventtrace": "1/5", 2020-11-16T20:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_filer": "0/1", 2020-11-16T20:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_filestore": "1/3", 2020-11-16T20:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_finisher": "1/1", 2020-11-16T20:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_fuse": "1/5", 2020-11-16T20:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_heartbeat_testing_span": "0", 2020-11-16T20:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_heartbeatmap": "1/5", 2020-11-16T20:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_immutable_obj_cache": "0/5", 2020-11-16T20:24:02.417 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_javaclient": "1/5", 2020-11-16T20:24:02.417 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_journal": "1/3", 2020-11-16T20:24:02.417 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_journaler": "0/5", 2020-11-16T20:24:02.417 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_kstore": "1/5", 2020-11-16T20:24:02.417 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_leveldb": "4/5", 2020-11-16T20:24:02.417 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_lockdep": "0/1", 2020-11-16T20:24:02.417 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds": "1/5", 2020-11-16T20:24:02.418 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_balancer": "1/5", 2020-11-16T20:24:02.418 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_locker": "1/5", 2020-11-16T20:24:02.418 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_log": "1/5", 2020-11-16T20:24:02.418 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_log_expire": "1/5", 2020-11-16T20:24:02.418 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_migrator": "1/5", 2020-11-16T20:24:02.418 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_memdb": "4/5", 2020-11-16T20:24:02.418 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mgr": "1/5", 2020-11-16T20:24:02.419 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mgrc": "1/5", 2020-11-16T20:24:02.419 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mon": "20/20", 2020-11-16T20:24:02.419 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_monc": "0/10", 2020-11-16T20:24:02.419 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_ms": "1/1", 2020-11-16T20:24:02.419 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_none": "0/5", 2020-11-16T20:24:02.419 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objclass": "0/5", 2020-11-16T20:24:02.419 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objectcacher": "0/5", 2020-11-16T20:24:02.419 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objecter": "0/1", 2020-11-16T20:24:02.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_optracker": "0/5", 2020-11-16T20:24:02.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_osd": "1/5", 2020-11-16T20:24:02.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_paxos": "20/20", 2020-11-16T20:24:02.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_perfcounter": "1/5", 2020-11-16T20:24:02.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_prioritycache": "1/5", 2020-11-16T20:24:02.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rados": "0/5", 2020-11-16T20:24:02.420 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd": "0/5", 2020-11-16T20:24:02.421 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_mirror": "0/5", 2020-11-16T20:24:02.421 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_pwl": "0/5", 2020-11-16T20:24:02.421 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_replay": "0/5", 2020-11-16T20:24:02.421 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_refs": "0/0", 2020-11-16T20:24:02.421 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_reserver": "1/1", 2020-11-16T20:24:02.421 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw": "1/5", 2020-11-16T20:24:02.421 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_sync": "1/5", 2020-11-16T20:24:02.422 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rocksdb": "4/5", 2020-11-16T20:24:02.422 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_striper": "0/1", 2020-11-16T20:24:02.422 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_test": "0/5", 2020-11-16T20:24:02.422 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_throttle": "1/1", 2020-11-16T20:24:02.422 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_timer": "0/1", 2020-11-16T20:24:02.422 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_tp": "0/5", 2020-11-16T20:24:02.422 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_version_for_testing": "", 2020-11-16T20:24:02.423 INFO:teuthology.orchestra.run.smithi092.stdout: "device_failure_prediction_mode": "none", 2020-11-16T20:24:02.423 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2020-11-16T20:24:02.423 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2020-11-16T20:24:02.423 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_graylog": "false", 2020-11-16T20:24:02.423 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_stderr": "true", 2020-11-16T20:24:02.423 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_syslog": "false", 2020-11-16T20:24:02.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event_tracing": "false", 2020-11-16T20:24:02.424 INFO:teuthology.orchestra.run.smithi092.stdout: "fake_statfs_for_testing": "0", 2020-11-16T20:24:02.424 INFO:teuthology.orchestra.run.smithi092.stdout: "fatal_signal_handlers": "true", 2020-11-16T20:24:02.424 INFO:teuthology.orchestra.run.smithi092.stdout: "filer_max_purge_ops": "10", 2020-11-16T20:24:02.424 INFO:teuthology.orchestra.run.smithi092.stdout: "filer_max_truncate_ops": "128", 2020-11-16T20:24:02.424 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_apply_finisher_threads": "1", 2020-11-16T20:24:02.424 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_blackhole": "false", 2020-11-16T20:24:02.424 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_btrfs_clone_range": "true", 2020-11-16T20:24:02.424 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_btrfs_snap": "true", 2020-11-16T20:24:02.425 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_caller_concurrency": "10", 2020-11-16T20:24:02.425 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_collect_device_partition_information": "true", 2020-11-16T20:24:02.425 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_commit_timeout": "600.000000", 2020-11-16T20:24:02.425 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_inject_read_err": "false", 2020-11-16T20:24:02.425 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_omap_check": "false", 2020-11-16T20:24:02.425 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_random_read_err": "0.000000", 2020-11-16T20:24:02.425 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_verify_split": "false", 2020-11-16T20:24:02.426 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_dump_file": "", 2020-11-16T20:24:02.426 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2020-11-16T20:24:02.426 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_expected_throughput_ops": "200.000000", 2020-11-16T20:24:02.426 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fadvise": "true", 2020-11-16T20:24:02.426 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fail_eio": "true", 2020-11-16T20:24:02.426 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fd_cache_shards": "16", 2020-11-16T20:24:02.426 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fd_cache_size": "128", 2020-11-16T20:24:02.426 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fiemap": "false", 2020-11-16T20:24:02.427 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fiemap_threshold": "4096", 2020-11-16T20:24:02.427 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fsync_flushes_journal_data": "false", 2020-11-16T20:24:02.427 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_index_retry_probability": "0.000000", 2020-11-16T20:24:02.427 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_inject_stall": "0", 2020-11-16T20:24:02.427 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_parallel": "false", 2020-11-16T20:24:02.427 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_trailing": "false", 2020-11-16T20:24:02.427 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_writeahead": "false", 2020-11-16T20:24:02.428 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_kill_at": "0", 2020-11-16T20:24:02.428 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_alloc_hint_size": "1048576", 2020-11-16T20:24:02.428 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size": "0", 2020-11-16T20:24:02.428 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2020-11-16T20:24:02.428 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_other": "512", 2020-11-16T20:24:02.428 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2020-11-16T20:24:02.428 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs": "0", 2020-11-16T20:24:02.429 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2020-11-16T20:24:02.429 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_other": "2", 2020-11-16T20:24:02.429 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_xfs": "10", 2020-11-16T20:24:02.429 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_sync_interval": "5.000000", 2020-11-16T20:24:02.429 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size": "0", 2020-11-16T20:24:02.429 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2020-11-16T20:24:02.429 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_other": "1024", 2020-11-16T20:24:02.430 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2020-11-16T20:24:02.430 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_merge_threshold": "-10", 2020-11-16T20:24:02.430 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_min_sync_interval": "0.010000", 2020-11-16T20:24:02.430 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_odsync_write": "false", 2020-11-16T20:24:02.430 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_backend": "rocksdb", 2020-11-16T20:24:02.430 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_backend_path": "", 2020-11-16T20:24:02.430 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_header_cache_size": "1024", 2020-11-16T20:24:02.431 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_ondisk_finisher_threads": "1", 2020-11-16T20:24:02.431 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_thread_suicide_timeout": "180", 2020-11-16T20:24:02.431 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_thread_timeout": "60", 2020-11-16T20:24:02.431 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_threads": "2", 2020-11-16T20:24:02.431 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_punch_hole": "false", 2020-11-16T20:24:02.431 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2020-11-16T20:24:02.431 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2020-11-16T20:24:02.431 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2020-11-16T20:24:02.432 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_threshhold": "0.900000", 2020-11-16T20:24:02.432 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_low_threshhold": "0.300000", 2020-11-16T20:24:02.432 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_bytes": "104857600", 2020-11-16T20:24:02.432 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2020-11-16T20:24:02.432 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2020-11-16T20:24:02.432 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2020-11-16T20:24:02.432 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_ops": "50", 2020-11-16T20:24:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2020-11-16T20:24:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_seek_data_hole": "false", 2020-11-16T20:24:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_sloppy_crc": "false", 2020-11-16T20:24:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_sloppy_crc_block_size": "65536", 2020-11-16T20:24:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_splice": "false", 2020-11-16T20:24:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_split_multiple": "2", 2020-11-16T20:24:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_split_rand_factor": "20", 2020-11-16T20:24:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_update_to": "1000", 2020-11-16T20:24:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2020-11-16T20:24:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2020-11-16T20:24:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2020-11-16T20:24:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2020-11-16T20:24:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2020-11-16T20:24:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2020-11-16T20:24:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_enable": "true", 2020-11-16T20:24:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2020-11-16T20:24:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2020-11-16T20:24:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2020-11-16T20:24:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2020-11-16T20:24:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2020-11-16T20:24:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2020-11-16T20:24:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_xfs_extsize": "false", 2020-11-16T20:24:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_zfs_snap": "false", 2020-11-16T20:24:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "fio_dir": "/tmp/fio", 2020-11-16T20:24:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "3fc8a474-4fd1-4d15-a656-0a82bbfce204", 2020-11-16T20:24:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_allow_other": "true", 2020-11-16T20:24:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_atomic_o_trunc": "true", 2020-11-16T20:24:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_big_writes": "true", 2020-11-16T20:24:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_debug": "false", 2020-11-16T20:24:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_default_permissions": "false", 2020-11-16T20:24:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_disable_pagecache": "false", 2020-11-16T20:24:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_max_write": "0", 2020-11-16T20:24:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_multithreaded": "true", 2020-11-16T20:24:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_require_active_mds": "true", 2020-11-16T20:24:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_set_user_groups": "true", 2020-11-16T20:24:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_move": "true", 2020-11-16T20:24:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_read": "true", 2020-11-16T20:24:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_write": "true", 2020-11-16T20:24:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_syncfs_on_mksnap": "true", 2020-11-16T20:24:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_use_invalidate_cb": "true", 2020-11-16T20:24:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2020-11-16T20:24:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "gss_target_name": "ceph", 2020-11-16T20:24:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_file": "", 2020-11-16T20:24:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_inject_failure": "0", 2020-11-16T20:24:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_interval": "5", 2020-11-16T20:24:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "host": "", 2020-11-16T20:24:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2020-11-16T20:24:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_max_inflight_ops": "128", 2020-11-16T20:24:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_max_size": "1073741824", 2020-11-16T20:24:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2020-11-16T20:24:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2020-11-16T20:24:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_watermark": "0.100000", 2020-11-16T20:24:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "inject_early_sigterm": "false", 2020-11-16T20:24:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_aio": "true", 2020-11-16T20:24:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_align_min_size": "65536", 2020-11-16T20:24:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_block_align": "true", 2020-11-16T20:24:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_block_size": "4096", 2020-11-16T20:24:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_dio": "true", 2020-11-16T20:24:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_discard": "false", 2020-11-16T20:24:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_force_aio": "false", 2020-11-16T20:24:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_ignore_corruption": "false", 2020-11-16T20:24:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_max_write_bytes": "10485760", 2020-11-16T20:24:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_max_write_entries": "100", 2020-11-16T20:24:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_replay_from": "0", 2020-11-16T20:24:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_high_multiple": "0.000000", 2020-11-16T20:24:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_high_threshhold": "0.900000", 2020-11-16T20:24:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_low_threshhold": "0.600000", 2020-11-16T20:24:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_max_multiple": "0.000000", 2020-11-16T20:24:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_write_header_frequency": "0", 2020-11-16T20:24:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_zero_on_create": "false", 2020-11-16T20:24:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_prefetch_periods": "10", 2020-11-16T20:24:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_prezero_periods": "5", 2020-11-16T20:24:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_write_head_interval": "15", 2020-11-16T20:24:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "", 2020-11-16T20:24:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "keyfile": "", 2020-11-16T20:24:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2020-11-16T20:24:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_backend": "rocksdb", 2020-11-16T20:24:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_default_stripe_size": "65536", 2020-11-16T20:24:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_fsck_on_mount": "false", 2020-11-16T20:24:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_fsck_on_mount_deep": "true", 2020-11-16T20:24:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_max_bytes": "67108864", 2020-11-16T20:24:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_max_ops": "512", 2020-11-16T20:24:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_nid_prealloc": "1024", 2020-11-16T20:24:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_onode_map_size": "1024", 2020-11-16T20:24:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2020-11-16T20:24:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_sync_submit_transaction": "false", 2020-11-16T20:24:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_sync_transaction": "false", 2020-11-16T20:24:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_block_size": "65536", 2020-11-16T20:24:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_bloom_size": "0", 2020-11-16T20:24:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_cache_size": "536870912", 2020-11-16T20:24:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_compact_on_mount": "false", 2020-11-16T20:24:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_compression": "false", 2020-11-16T20:24:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_log": "", 2020-11-16T20:24:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_log_to_ceph_log": "true", 2020-11-16T20:24:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_max_open_files": "0", 2020-11-16T20:24:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_paranoid": "false", 2020-11-16T20:24:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "leveldb_write_buffer_size": "33554432", 2020-11-16T20:24:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "librados_thread_count": "2", 2020-11-16T20:24:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "lockdep": "false", 2020-11-16T20:24:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "lockdep_force_backtrace": "false", 2020-11-16T20:24:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "log_coarse_timestamps": "true", 2020-11-16T20:24:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2020-11-16T20:24:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_flush_on_exit": "false", 2020-11-16T20:24:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_graylog_host": "127.0.0.1", 2020-11-16T20:24:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_graylog_port": "12201", 2020-11-16T20:24:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_max_new": "1000", 2020-11-16T20:24:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_max_recent": "10000", 2020-11-16T20:24:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_stderr_prefix": "", 2020-11-16T20:24:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "log_stop_at_utilization": "0.970000", 2020-11-16T20:24:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": "true", 2020-11-16T20:24:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_graylog": "false", 2020-11-16T20:24:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_stderr": "false", 2020-11-16T20:24:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_syslog": "false", 2020-11-16T20:24:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "max_rotating_auth_attempts": "10", 2020-11-16T20:24:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_action_on_write_error": "1", 2020-11-16T20:24:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_asio_thread_count": "2", 2020-11-16T20:24:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_export_pin": "true", 2020-11-16T20:24:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_dirs": "true", 2020-11-16T20:24:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2020-11-16T20:24:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_interval": "5", 2020-11-16T20:24:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_size_max": "100000", 2020-11-16T20:24:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_idle_threshold": "0.000000", 2020-11-16T20:24:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_interval": "10", 2020-11-16T20:24:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_max": "-1", 2020-11-16T20:24:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_max_until": "-1", 2020-11-16T20:24:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_merge_size": "50", 2020-11-16T20:24:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_midchunk": "0.300000", 2020-11-16T20:24:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_min_rebalance": "0.100000", 2020-11-16T20:24:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_min_start": "0.200000", 2020-11-16T20:24:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_minchunk": "0.001000", 2020-11-16T20:24:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_mode": "0", 2020-11-16T20:24:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_need_max": "1.200000", 2020-11-16T20:24:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_need_min": "0.800000", 2020-11-16T20:24:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_replicate_threshold": "8000.000000", 2020-11-16T20:24:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_sample_interval": "3.000000", 2020-11-16T20:24:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_bits": "3", 2020-11-16T20:24:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_rd": "25000.000000", 2020-11-16T20:24:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_size": "10000", 2020-11-16T20:24:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_wr": "10000.000000", 2020-11-16T20:24:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_target_decay": "10.000000", 2020-11-16T20:24:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2020-11-16T20:24:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_grace": "15.000000", 2020-11-16T20:24:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_interval": "4.000000", 2020-11-16T20:24:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_memory_limit": "4294967296", 2020-11-16T20:24:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_mid": "0.700000", 2020-11-16T20:24:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_release_free_interval": "10", 2020-11-16T20:24:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_reservation": "0.050000", 2020-11-16T20:24:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_decay_rate": "1.000000", 2020-11-16T20:24:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_interval": "1", 2020-11-16T20:24:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_threshold": "65536", 2020-11-16T20:24:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2020-11-16T20:24:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2020-11-16T20:24:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_delegate_inos_pct": "50", 2020-11-16T20:24:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_prealloc_inos": "1000", 2020-11-16T20:24:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2020-11-16T20:24:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_damage_table_max_entries": "10000", 2020-11-16T20:24:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2020-11-16T20:24:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_auth_pins": "false", 2020-11-16T20:24:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_frag": "false", 2020-11-16T20:24:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_scatterstat": "false", 2020-11-16T20:24:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_subtrees": "false", 2020-11-16T20:24:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_decay_halflife": "5.000000", 2020-11-16T20:24:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_default_dir_hash": "2", 2020-11-16T20:24:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_defer_session_stale": "true", 2020-11-16T20:24:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_deny_all_reconnect": "false", 2020-11-16T20:24:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_keys_per_op": "16384", 2020-11-16T20:24:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_max_commit_size": "10", 2020-11-16T20:24:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dirstat_min_interval": "1.000000", 2020-11-16T20:24:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_after_rejoin": "false", 2020-11-16T20:24:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_on_map": "false", 2020-11-16T20:24:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_threshold_file": "0", 2020-11-16T20:24:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2020-11-16T20:24:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_early_reply": "true", 2020-11-16T20:24:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_enable_op_tracker": "true", 2020-11-16T20:24:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_enforce_unique_name": "true", 2020-11-16T20:24:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_distributed": "true", 2020-11-16T20:24:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2020-11-16T20:24:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_random": "true", 2020-11-16T20:24:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_random_max": "0.010000", 2020-11-16T20:24:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_forward_all_requests_to_auth": "false", 2020-11-16T20:24:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_freeze_tree_timeout": "30.000000", 2020-11-16T20:24:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2020-11-16T20:24:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_health_cache_threshold": "1.500000", 2020-11-16T20:24:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_health_summarize_threshold": "10", 2020-11-16T20:24:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_heartbeat_grace": "15.000000", 2020-11-16T20:24:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_migrator_session_race": "false", 2020-11-16T20:24:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2020-11-16T20:24:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_join_fs": "", 2020-11-16T20:24:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_journal_format": "1", 2020-11-16T20:24:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_create_at": "0", 2020-11-16T20:24:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_export_at": "0", 2020-11-16T20:24:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_import_at": "0", 2020-11-16T20:24:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_at": "0", 2020-11-16T20:24:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_expire_at": "0", 2020-11-16T20:24:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_replay_at": "0", 2020-11-16T20:24:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_link_at": "0", 2020-11-16T20:24:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_mdstable_at": "0", 2020-11-16T20:24:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_openc_at": "0", 2020-11-16T20:24:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_rename_at": "0", 2020-11-16T20:24:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_events_per_segment": "1024", 2020-11-16T20:24:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_max_events": "-1", 2020-11-16T20:24:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_max_segments": "128", 2020-11-16T20:24:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_pause": "false", 2020-11-16T20:24:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_segment_size": "0", 2020-11-16T20:24:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_skip_corrupt_events": "false", 2020-11-16T20:24:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_warn_factor": "2.000000", 2020-11-16T20:24:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_caps_per_client": "1048576", 2020-11-16T20:24:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_completed_flushes": "100000", 2020-11-16T20:24:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_completed_requests": "100000", 2020-11-16T20:24:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_export_size": "20971520", 2020-11-16T20:24:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_file_recover": "32", 2020-11-16T20:24:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_files": "64", 2020-11-16T20:24:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_ops": "8192", 2020-11-16T20:24:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2020-11-16T20:24:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_retries_on_remount_failure": "5", 2020-11-16T20:24:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_scrub_ops_in_progress": "5", 2020-11-16T20:24:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_snaps_per_dir": "100", 2020-11-16T20:24:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_xattr_pairs_size": "65536", 2020-11-16T20:24:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_metrics_update_interval": "2", 2020-11-16T20:24:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_min_caps_per_client": "100", 2020-11-16T20:24:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_min_caps_working_set": "10000", 2020-11-16T20:24:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mon_shutdown_timeout": "5.000000", 2020-11-16T20:24:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_numa_node": "-1", 2020-11-16T20:24:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_oft_prefetch_dirfrags": "true", 2020-11-16T20:24:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_complaint_time": "30.000000", 2020-11-16T20:24:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_duration": "600", 2020-11-16T20:24:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_size": "20", 2020-11-16T20:24:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_log_threshold": "5", 2020-11-16T20:24:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_ping_grace": "15", 2020-11-16T20:24:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_ping_interval": "5", 2020-11-16T20:24:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2020-11-16T20:24:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_global_max_decay_threshold": "65536", 2020-11-16T20:24:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_caps": "5000", 2020-11-16T20:24:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_decay_rate": "2.500000", 2020-11-16T20:24:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_decay_threshold": "16384", 2020-11-16T20:24:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_warning_decay_rate": "60.000000", 2020-11-16T20:24:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_warning_threshold": "32768", 2020-11-16T20:24:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_reconnect_timeout": "45.000000", 2020-11-16T20:24:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_replay_interval": "1.000000", 2020-11-16T20:24:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_replay_unsafe_with_closed_session": "false", 2020-11-16T20:24:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_request_load_average_decay_rate": "60.000000", 2020-11-16T20:24:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_root_ino_gid": "0", 2020-11-16T20:24:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_root_ino_uid": "0", 2020-11-16T20:24:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_scatter_nudge_interval": "5.000000", 2020-11-16T20:24:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_blocklist_on_evict": "true", 2020-11-16T20:24:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_blocklist_on_timeout": "true", 2020-11-16T20:24:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2020-11-16T20:24:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cache_liveness_magnitude": "10", 2020-11-16T20:24:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2020-11-16T20:24:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cap_acquisition_throttle": "500000", 2020-11-16T20:24:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2020-11-16T20:24:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessionmap_keys_per_op": "1024", 2020-11-16T20:24:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_shutdown_check": "0", 2020-11-16T20:24:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_skip_ino": "0", 2020-11-16T20:24:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_max_uid": "4294967294", 2020-11-16T20:24:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_min_uid": "0", 2020-11-16T20:24:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_rstat": "false", 2020-11-16T20:24:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_task_status_update_interval": "2.000000", 2020-11-16T20:24:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_thrash_exports": "0", 2020-11-16T20:24:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_thrash_fragments": "0", 2020-11-16T20:24:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_tick_interval": "5.000000", 2020-11-16T20:24:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_verify_backtrace": "1", 2020-11-16T20:24:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_verify_scatter": "false", 2020-11-16T20:24:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_wipe_ino_prealloc": "false", 2020-11-16T20:24:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_wipe_sessions": "false", 2020-11-16T20:24:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool_debug": "false", 2020-11-16T20:24:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_debug_omit_block_device_write": "false", 2020-11-16T20:24:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_device_bytes": "1073741824", 2020-11-16T20:24:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_page_set": "false", 2020-11-16T20:24:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_page_size": "65536", 2020-11-16T20:24:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_bytes": "134217728", 2020-11-16T20:24:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_messages": "512", 2020-11-16T20:24:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2020-11-16T20:24:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_connect_retry_interval": "1.000000", 2020-11-16T20:24:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2020-11-16T20:24:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2020-11-16T20:24:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_disabled_modules": "", 2020-11-16T20:24:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_initial_modules": "restful iostat", 2020-11-16T20:24:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mds_bytes": "134217728", 2020-11-16T20:24:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mds_messages": "128", 2020-11-16T20:24:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2020-11-16T20:24:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mon_bytes": "134217728", 2020-11-16T20:24:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mon_messages": "128", 2020-11-16T20:24:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_osd_bytes": "536870912", 2020-11-16T20:24:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_osd_messages": "8192", 2020-11-16T20:24:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_service_beacon_grace": "60.000000", 2020-11-16T20:24:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_stats_period": "5", 2020-11-16T20:24:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_stats_threshold": "5", 2020-11-16T20:24:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_tick_period": "2", 2020-11-16T20:24:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_accept_timeout_factor": "2.000000", 2020-11-16T20:24:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_allow_pool_delete": "true", 2020-11-16T20:24:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_allow_pool_size_one": "true", 2020-11-16T20:24:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_auth_validate_all_caps": "true", 2020-11-16T20:24:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2020-11-16T20:24:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2020-11-16T20:24:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_bytes": "104857600", 2020-11-16T20:24:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_directed_command_retry": "5", 2020-11-16T20:24:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval": "3.000000", 2020-11-16T20:24:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2020-11-16T20:24:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2020-11-16T20:24:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2020-11-16T20:24:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_parallel": "3", 2020-11-16T20:24:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_log_interval": "1.000000", 2020-11-16T20:24:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_max_log_entries_per_message": "1000", 2020-11-16T20:24:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_ping_interval": "10.000000", 2020-11-16T20:24:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_ping_timeout": "30.000000", 2020-11-16T20:24:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clock_drift_allowed": "1.000000", 2020-11-16T20:24:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2020-11-16T20:24:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2020-11-16T20:24:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_file_level": "debug", 2020-11-16T20:24:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_file": "true", 2020-11-16T20:24:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog": "false", 2020-11-16T20:24:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2020-11-16T20:24:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog_port": "12201", 2020-11-16T20:24:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_stderr": "false", 2020-11-16T20:24:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog": "default=false", 2020-11-16T20:24:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2020-11-16T20:24:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog_level": "info", 2020-11-16T20:24:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_bootstrap": "false", 2020-11-16T20:24:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_start": "false", 2020-11-16T20:24:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_trim": "true", 2020-11-16T20:24:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_con_tracker_persist_interval": "10", 2020-11-16T20:24:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_con_tracker_score_halflife": "43200", 2020-11-16T20:24:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_config_key_max_entry_size": "65536", 2020-11-16T20:24:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cpu_threads": "4", 2020-11-16T20:24:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_crush_min_required_version": "hammer", 2020-11-16T20:24:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_daemon_bytes": "419430400", 2020-11-16T20:24:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2020-11-16T20:24:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_avail_crit": "5", 2020-11-16T20:24:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_avail_warn": "5", 2020-11-16T20:24:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_size_warn": "16106127360", 2020-11-16T20:24:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_block_osdmap_trim": "false", 2020-11-16T20:24:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_deprecated_as_obsolete": "false", 2020-11-16T20:24:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_json": "false", 2020-11-16T20:24:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2020-11-16T20:24:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_transactions": "false", 2020-11-16T20:24:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_extra_checks": "false", 2020-11-16T20:24:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_initial_persistent_features": "false", 2020-11-16T20:24:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2020-11-16T20:24:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_octopus": "false", 2020-11-16T20:24:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_pacific": "false", 2020-11-16T20:24:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2020-11-16T20:24:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_delta_reset_interval": "10.000000", 2020-11-16T20:24:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_dns_srv_name": "ceph-mon", 2020-11-16T20:24:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_election_default_strategy": "1", 2020-11-16T20:24:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_election_timeout": "5.000000", 2020-11-16T20:24:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2020-11-16T20:24:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ping_divisor": "2", 2020-11-16T20:24:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ping_timeout": "2.000000", 2020-11-16T20:24:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_enable_op_tracker": "true", 2020-11-16T20:24:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_fake_pool_delete": "false", 2020-11-16T20:24:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_force_quorum_join": "false", 2020-11-16T20:24:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_globalid_prealloc": "10000", 2020-11-16T20:24:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_detail_to_clog": "false", 2020-11-16T20:24:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_log_update_period": "5", 2020-11-16T20:24:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_max_detail": "50", 2020-11-16T20:24:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog": "true", 2020-11-16T20:24:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog_interval": "600", 2020-11-16T20:24:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2020-11-16T20:24:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_host": "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.13", 2020-11-16T20:24:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_host_override": "", 2020-11-16T20:24:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_initial_members": "", 2020-11-16T20:24:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2020-11-16T20:24:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2020-11-16T20:24:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_transaction_delay_max": "10.000000", 2020-11-16T20:24:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2020-11-16T20:24:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_keyvaluedb": "rocksdb", 2020-11-16T20:24:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease": "5.000000", 2020-11-16T20:24:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2020-11-16T20:24:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease_renew_interval_factor": "0.600000", 2020-11-16T20:24:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_max_summary": "50", 2020-11-16T20:24:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_log_entries_per_event": "4096", 2020-11-16T20:24:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_log_epochs": "500", 2020-11-16T20:24:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_mdsmap_epochs": "500", 2020-11-16T20:24:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_mgrmap_epochs": "500", 2020-11-16T20:24:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_osd": "10000", 2020-11-16T20:24:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_pg_per_osd": "10000", 2020-11-16T20:24:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_pool_pg_num": "65536", 2020-11-16T20:24:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_snap_prune_per_epoch": "100", 2020-11-16T20:24:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_blocklist_interval": "86400.000000", 2020-11-16T20:24:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_force_trim_to": "0", 2020-11-16T20:24:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_skip_sanity": "false", 2020-11-16T20:24:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_memory_autotune": "true", 2020-11-16T20:24:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_memory_target": "2147483648", 2020-11-16T20:24:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_beacon_grace": "30", 2020-11-16T20:24:02.487 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2020-11-16T20:24:02.487 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_digest_period": "5", 2020-11-16T20:24:02.487 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_inactive_grace": "60", 2020-11-16T20:24:02.487 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_mkfs_grace": "240", 2020-11-16T20:24:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2020-11-16T20:24:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_min_osdmap_epochs": "50", 2020-11-16T20:24:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_complaint_time": "30", 2020-11-16T20:24:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_duration": "600", 2020-11-16T20:24:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_size": "20", 2020-11-16T20:24:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_slow_op_size": "20", 2020-11-16T20:24:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_slow_op_threshold": "10", 2020-11-16T20:24:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_log_threshold": "5", 2020-11-16T20:24:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_adjust_down_out_interval": "true", 2020-11-16T20:24:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2020-11-16T20:24:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2020-11-16T20:24:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_in": "false", 2020-11-16T20:24:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_new_in": "true", 2020-11-16T20:24:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2020-11-16T20:24:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2020-11-16T20:24:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_cache_size": "500", 2020-11-16T20:24:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_cache_size_min": "134217728", 2020-11-16T20:24:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_crush_smoke_test": "true", 2020-11-16T20:24:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_destroyed_out_interval": "600", 2020-11-16T20:24:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_down_out_interval": "600", 2020-11-16T20:24:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_down_out_subtree_limit": "rack", 2020-11-16T20:24:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2020-11-16T20:24:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_force_trim_to": "0", 2020-11-16T20:24:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_full_ratio": "0.950000", 2020-11-16T20:24:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2020-11-16T20:24:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_halflife": "3600", 2020-11-16T20:24:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_max_interval": "300", 2020-11-16T20:24:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_weight": "0.300000", 2020-11-16T20:24:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2020-11-16T20:24:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_max_creating_pgs": "1024", 2020-11-16T20:24:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_max_initial_pgs": "1024", 2020-11-16T20:24:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_down_reporters": "2", 2020-11-16T20:24:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_in_ratio": "0.750000", 2020-11-16T20:24:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_up_ratio": "0.300000", 2020-11-16T20:24:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_nearfull_ratio": "0.850000", 2020-11-16T20:24:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp": "true", 2020-11-16T20:24:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2020-11-16T20:24:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2020-11-16T20:24:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_report_timeout": "900", 2020-11-16T20:24:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_reporter_subtree_level": "osd", 2020-11-16T20:24:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2020-11-16T20:24:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_warn_num_repaired": "10", 2020-11-16T20:24:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_warn_op_age": "32.000000", 2020-11-16T20:24:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_enabled": "true", 2020-11-16T20:24:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_interval": "2", 2020-11-16T20:24:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_min": "15", 2020-11-16T20:24:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_txsize": "2", 2020-11-16T20:24:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2020-11-16T20:24:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_stuck_threshold": "60", 2020-11-16T20:24:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2020-11-16T20:24:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_objects": "10000", 2020-11-16T20:24:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_per_osd": "0", 2020-11-16T20:24:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_pool_objects": "1000", 2020-11-16T20:24:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pool_quota_crit_threshold": "0", 2020-11-16T20:24:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pool_quota_warn_threshold": "0", 2020-11-16T20:24:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_probe_timeout": "2.000000", 2020-11-16T20:24:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_max_change": "0.050000", 2020-11-16T20:24:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_max_osds": "4", 2020-11-16T20:24:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_min_bytes_per_osd": "10", 2020-11-16T20:24:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_min_pgs_per_osd": "4", 2020-11-16T20:24:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2020-11-16T20:24:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2020-11-16T20:24:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2020-11-16T20:24:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_interval": "86400", 2020-11-16T20:24:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_max_keys": "100", 2020-11-16T20:24:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_timeout": "300", 2020-11-16T20:24:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_session_timeout": "300", 2020-11-16T20:24:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_smart_report_timeout": "5", 2020-11-16T20:24:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stat_smooth_intervals": "6", 2020-11-16T20:24:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2020-11-16T20:24:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_pool_min_size": "2", 2020-11-16T20:24:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_pool_size": "4", 2020-11-16T20:24:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2020-11-16T20:24:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_subscribe_interval": "86400.000000", 2020-11-16T20:24:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_debug": "false", 2020-11-16T20:24:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_max_payload_keys": "2000", 2020-11-16T20:24:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_max_payload_size": "1048576", 2020-11-16T20:24:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_provider_kill_at": "0", 2020-11-16T20:24:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_requester_kill_at": "0", 2020-11-16T20:24:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_timeout": "60.000000", 2020-11-16T20:24:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_target_pg_per_osd": "100", 2020-11-16T20:24:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_tick_interval": "5", 2020-11-16T20:24:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_timecheck_interval": "300.000000", 2020-11-16T20:24:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_timecheck_skew_interval": "30.000000", 2020-11-16T20:24:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_older_version_delay": "604800.000000", 2020-11-16T20:24:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2020-11-16T20:24:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2020-11-16T20:24:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2020-11-16T20:24:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2020-11-16T20:24:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_misplaced": "false", 2020-11-16T20:24:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2020-11-16T20:24:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_older_version": "true", 2020-11-16T20:24:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2020-11-16T20:24:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_no_app": "true", 2020-11-16T20:24:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_no_redundancy": "false", 2020-11-16T20:24:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2020-11-16T20:24:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2020-11-16T20:24:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2020-11-16T20:24:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_too_few_osds": "false", 2020-11-16T20:24:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2020-11-16T20:24:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2020-11-16T20:24:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "monmap": "", 2020-11-16T20:24:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_max_op_threads": "5", 2020-11-16T20:24:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_op_threads": "3", 2020-11-16T20:24:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_buffer_size": "131072", 2020-11-16T20:24:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_cm": "false", 2020-11-16T20:24:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_device_name": "", 2020-11-16T20:24:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_dscp": "96", 2020-11-16T20:24:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_enable_hugepage": "false", 2020-11-16T20:24:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_gid_idx": "0", 2020-11-16T20:24:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_local_gid": "", 2020-11-16T20:24:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_polling_us": "1000", 2020-11-16T20:24:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_port_num": "1", 2020-11-16T20:24:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_receive_buffers": "32768", 2020-11-16T20:24:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_receive_queue_len": "4096", 2020-11-16T20:24:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_roce_ver": "1", 2020-11-16T20:24:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_send_buffers": "1024", 2020-11-16T20:24:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_sl": "3", 2020-11-16T20:24:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_support_srq": "true", 2020-11-16T20:24:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_type": "ib", 2020-11-16T20:24:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_before_connect": "false", 2020-11-16T20:24:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_ipv4": "true", 2020-11-16T20:24:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_ipv6": "false", 2020-11-16T20:24:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_msgr1": "true", 2020-11-16T20:24:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_msgr2": "true", 2020-11-16T20:24:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_port_max": "7300", 2020-11-16T20:24:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_port_min": "6800", 2020-11-16T20:24:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_prefer_ipv4": "false", 2020-11-16T20:24:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_retry_count": "3", 2020-11-16T20:24:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_retry_delay": "5", 2020-11-16T20:24:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_client": "false", 2020-11-16T20:24:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mds": "false", 2020-11-16T20:24:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mgr": "false", 2020-11-16T20:24:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mon": "false", 2020-11-16T20:24:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_osd": "false", 2020-11-16T20:24:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_client_mode": "crc secure", 2020-11-16T20:24:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_cluster_mode": "crc secure", 2020-11-16T20:24:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_cluster_type": "", 2020-11-16T20:24:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_connection_idle_timeout": "900", 2020-11-16T20:24:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_connection_ready_timeout": "10", 2020-11-16T20:24:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_crc_data": "true", 2020-11-16T20:24:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_crc_header": "true", 2020-11-16T20:24:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_bad_msg": "false", 2020-11-16T20:24:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_bug": "true", 2020-11-16T20:24:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_old_message": "true", 2020-11-16T20:24:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_skipped_message": "false", 2020-11-16T20:24:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_unhandled_msg": "false", 2020-11-16T20:24:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dispatch_throttle_bytes": "104857600", 2020-11-16T20:24:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_coremask": "0xF", 2020-11-16T20:24:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_debug_allow_loopback": "false", 2020-11-16T20:24:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2020-11-16T20:24:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_host_ipv4_addr": "", 2020-11-16T20:24:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hugepages": "", 2020-11-16T20:24:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hw_flow_control": "true", 2020-11-16T20:24:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2020-11-16T20:24:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_lro": "true", 2020-11-16T20:24:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_memory_channel": "4", 2020-11-16T20:24:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2020-11-16T20:24:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_pmd": "", 2020-11-16T20:24:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_port_id": "0", 2020-11-16T20:24:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2020-11-16T20:24:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dump_corrupt_message_level": "1", 2020-11-16T20:24:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dump_on_send": "false", 2020-11-16T20:24:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_initial_backoff": "0.200000", 2020-11-16T20:24:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_max": "1.000000", 2020-11-16T20:24:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_msg_type": "", 2020-11-16T20:24:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_probability": "0.005000", 2020-11-16T20:24:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_type": "osd", 2020-11-16T20:24:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_internal_delays": "0.002000", 2020-11-16T20:24:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_socket_failures": "2500", 2020-11-16T20:24:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_learn_addr_from_peer": "true", 2020-11-16T20:24:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_max_accept_failures": "4", 2020-11-16T20:24:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_max_backoff": "15.000000", 2020-11-16T20:24:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_client_mode": "secure crc", 2020-11-16T20:24:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_cluster_mode": "secure crc", 2020-11-16T20:24:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_service_mode": "secure crc", 2020-11-16T20:24:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2020-11-16T20:24:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_pq_min_cost": "65536", 2020-11-16T20:24:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_public_type": "", 2020-11-16T20:24:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_service_mode": "crc secure", 2020-11-16T20:24:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_listen_backlog": "512", 2020-11-16T20:24:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_nodelay": "true", 2020-11-16T20:24:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_prefetch_max_size": "4096", 2020-11-16T20:24:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_rcvbuf": "0", 2020-11-16T20:24:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_type": "async+posix", 2020-11-16T20:24:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: "no_config_file": "false", 2020-11-16T20:24:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_completion_locks_per_session": "32", 2020-11-16T20:24:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_debug_inject_relock_delay": "false", 2020-11-16T20:24:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inflight_op_bytes": "104857600", 2020-11-16T20:24:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inflight_ops": "1024", 2020-11-16T20:24:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inject_no_watch_ping": "false", 2020-11-16T20:24:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_retry_writes_after_first_reply": "false", 2020-11-16T20:24:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_tick_interval": "5.000000", 2020-11-16T20:24:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_timeout": "10.000000", 2020-11-16T20:24:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: "objectstore_blackhole": "false", 2020-11-16T20:24:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: "openssl_engine_opts": "", 2020-11-16T20:24:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_delay_time": "5.000000", 2020-11-16T20:24:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_hist_halflife": "1000", 2020-11-16T20:24:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_max_low_ops": "2", 2020-11-16T20:24:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_max_ops": "4", 2020-11-16T20:24:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_min_evict_effort": "0.100000", 2020-11-16T20:24:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_quantize_effort": "0.100000", 2020-11-16T20:24:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_slop": "0.020000", 2020-11-16T20:24:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_allow_recovery_below_min_size": "true", 2020-11-16T20:24:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_asio_thread_count": "2", 2020-11-16T20:24:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_async_recovery_min_cost": "1", 2020-11-16T20:24:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_auto_mark_unfound_lost": "false", 2020-11-16T20:24:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_retry_interval": "30.000000", 2020-11-16T20:24:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_scan_max": "512", 2020-11-16T20:24:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_scan_min": "64", 2020-11-16T20:24:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_degraded": "false", 2020-11-16T20:24:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_peering": "false", 2020-11-16T20:24:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_unfound": "true", 2020-11-16T20:24:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_beacon_report_interval": "300", 2020-11-16T20:24:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_duration": "30", 2020-11-16T20:24:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_large_size_max_throughput": "104857600", 2020-11-16T20:24:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_max_block_size": "67108864", 2020-11-16T20:24:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_small_size_max_iops": "100", 2020-11-16T20:24:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_blkin_trace_all": "false", 2020-11-16T20:24:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2020-11-16T20:24:02.536 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2020-11-16T20:24:02.536 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_check_for_log_corruption": "false", 2020-11-16T20:24:02.536 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_check_max_object_name_len_on_startup": "true", 2020-11-16T20:24:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2020-11-16T20:24:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2020-11-16T20:24:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2020-11-16T20:24:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_update_on_start": "true", 2020-11-16T20:24:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_message_cap": "0", 2020-11-16T20:24:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_message_size_cap": "524288000", 2020-11-16T20:24:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_op_priority": "63", 2020-11-16T20:24:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_watch_timeout": "30", 2020-11-16T20:24:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_command_max_records": "256", 2020-11-16T20:24:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_command_thread_suicide_timeout": "900", 2020-11-16T20:24:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_command_thread_timeout": "600", 2020-11-16T20:24:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_compact_on_start": "false", 2020-11-16T20:24:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_copyfrom_max_chunk": "8388608", 2020-11-16T20:24:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_chooseleaf_type": "0", 2020-11-16T20:24:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_initial_weight": "-1.000000", 2020-11-16T20:24:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_update_on_start": "true", 2020-11-16T20:24:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_update_weight_set": "true", 2020-11-16T20:24:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2020-11-16T20:24:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2020-11-16T20:24:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2020-11-16T20:24:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_drop_ping_duration": "0", 2020-11-16T20:24:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_drop_ping_probability": "0.000000", 2020-11-16T20:24:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_feed_pullee": "-1", 2020-11-16T20:24:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_copyfrom_error": "false", 2020-11-16T20:24:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2020-11-16T20:24:02.544 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2020-11-16T20:24:02.544 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_misdirected_ops": "false", 2020-11-16T20:24:02.544 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_no_acting_change": "false", 2020-11-16T20:24:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_no_purge_strays": "false", 2020-11-16T20:24:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_op_order": "false", 2020-11-16T20:24:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_pg_log_writeout": "false", 2020-11-16T20:24:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_pretend_recovery_active": "false", 2020-11-16T20:24:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_random_push_read_error": "0.000000", 2020-11-16T20:24:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2020-11-16T20:24:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_shutdown": "false", 2020-11-16T20:24:02.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2020-11-16T20:24:02.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2020-11-16T20:24:02.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_cached_snaps": "false", 2020-11-16T20:24:02.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_missing_on_start": "false", 2020-11-16T20:24:02.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_snaps": "false", 2020-11-16T20:24:02.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_stray_on_activate": "false", 2020-11-16T20:24:02.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_interval": "604800.000000", 2020-11-16T20:24:02.549 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_keys": "1024", 2020-11-16T20:24:02.549 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2020-11-16T20:24:02.549 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2020-11-16T20:24:02.549 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2020-11-16T20:24:02.550 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_stride": "524288", 2020-11-16T20:24:02.550 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2020-11-16T20:24:02.550 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_default_data_pool_replay_window": "5", 2020-11-16T20:24:02.551 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_default_notify_timeout": "30", 2020-11-16T20:24:02.551 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep": "0.000000", 2020-11-16T20:24:02.551 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_hdd": "5.000000", 2020-11-16T20:24:02.551 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_hybrid": "2.000000", 2020-11-16T20:24:02.552 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_ssd": "0.000000", 2020-11-16T20:24:02.552 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_discard_disconnected_ops": "true", 2020-11-16T20:24:02.552 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_enable_op_tracker": "true", 2020-11-16T20:24:02.552 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2020-11-16T20:24:02.553 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_failsafe_full_ratio": "0.970000", 2020-11-16T20:24:02.553 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_fail_on_connection_refused": "true", 2020-11-16T20:24:02.553 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_info": "true", 2020-11-16T20:24:02.553 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown": "true", 2020-11-16T20:24:02.553 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_find_best_info_ignore_history_les": "false", 2020-11-16T20:24:02.554 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_force_auth_primary_missing_objects": "100", 2020-11-16T20:24:02.554 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2020-11-16T20:24:02.554 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_function_tracing": "false", 2020-11-16T20:24:02.554 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_grace": "20", 2020-11-16T20:24:02.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_interval": "6", 2020-11-16T20:24:02.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2020-11-16T20:24:02.555 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_peers": "10", 2020-11-16T20:24:02.556 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_size": "2000", 2020-11-16T20:24:02.556 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_stale": "600", 2020-11-16T20:24:02.556 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2020-11-16T20:24:02.556 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_max_size": "100000", 2020-11-16T20:24:02.557 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_min_size": "1000", 2020-11-16T20:24:02.557 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_namespace": ".ceph-internal", 2020-11-16T20:24:02.557 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_ignore_stale_divergent_priors": "false", 2020-11-16T20:24:02.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2020-11-16T20:24:02.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_inject_failure_on_pg_removal": "false", 2020-11-16T20:24:02.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2020-11-16T20:24:02.558 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal_flush_on_shutdown": "true", 2020-11-16T20:24:02.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal_size": "5120", 2020-11-16T20:24:02.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_kill_backfill_at": "0", 2020-11-16T20:24:02.559 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_loop_before_reset_tphandle": "64", 2020-11-16T20:24:02.560 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_cache_size": "50", 2020-11-16T20:24:02.560 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_dedup": "true", 2020-11-16T20:24:02.560 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_message_max": "40", 2020-11-16T20:24:02.560 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_message_max_bytes": "10485760", 2020-11-16T20:24:02.560 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_share_max_epochs": "40", 2020-11-16T20:24:02.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_attr_name_len": "100", 2020-11-16T20:24:02.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_attr_size": "0", 2020-11-16T20:24:02.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_backfills": "1", 2020-11-16T20:24:02.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_markdown_count": "5", 2020-11-16T20:24:02.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_markdown_period": "600", 2020-11-16T20:24:02.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_name_len": "2048", 2020-11-16T20:24:02.561 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_namespace_len": "256", 2020-11-16T20:24:02.562 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_size": "134217728", 2020-11-16T20:24:02.562 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2020-11-16T20:24:02.562 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_omap_entries_per_request": "1024", 2020-11-16T20:24:02.562 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_blocked_by": "16", 2020-11-16T20:24:02.562 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_log_entries": "10000", 2020-11-16T20:24:02.562 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2020-11-16T20:24:02.562 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pgls": "1024", 2020-11-16T20:24:02.563 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_push_cost": "8388608", 2020-11-16T20:24:02.563 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_push_objects": "10", 2020-11-16T20:24:02.563 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_scrubs": "1", 2020-11-16T20:24:02.563 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2020-11-16T20:24:02.563 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_trimming_pgs": "2", 2020-11-16T20:24:02.563 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_write_op_reply_len": "32", 2020-11-16T20:24:02.563 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_write_size": "90", 2020-11-16T20:24:02.563 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2020-11-16T20:24:02.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2020-11-16T20:24:02.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2020-11-16T20:24:02.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2020-11-16T20:24:02.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2020-11-16T20:24:02.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2020-11-16T20:24:02.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2020-11-16T20:24:02.564 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_lim": "999999", 2020-11-16T20:24:02.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_res": "1", 2020-11-16T20:24:02.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_wgt": "1", 2020-11-16T20:24:02.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_base": "805306368", 2020-11-16T20:24:02.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_cache_min": "134217728", 2020-11-16T20:24:02.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_cache_resize_interval": "1.000000", 2020-11-16T20:24:02.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_expected_fragmentation": "0.150000", 2020-11-16T20:24:02.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target": "4294967296", 2020-11-16T20:24:02.565 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2020-11-16T20:24:02.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_min_pg_log_entries": "250", 2020-11-16T20:24:02.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_min_recovery_priority": "0", 2020-11-16T20:24:02.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_ack_timeout": "30.000000", 2020-11-16T20:24:02.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_heartbeat_interval": "30", 2020-11-16T20:24:02.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2020-11-16T20:24:02.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_report_interval": "5", 2020-11-16T20:24:02.566 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_report_max_in_flight": "2", 2020-11-16T20:24:02.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_shutdown_timeout": "5.000000", 2020-11-16T20:24:02.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_num_cache_shards": "32", 2020-11-16T20:24:02.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_num_op_tracker_shard": "32", 2020-11-16T20:24:02.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_auto_affinity": "true", 2020-11-16T20:24:02.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_node": "-1", 2020-11-16T20:24:02.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_prefer_iface": "true", 2020-11-16T20:24:02.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_object_clean_region_max_num_intervals": "10", 2020-11-16T20:24:02.567 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objecter_finishers": "1", 2020-11-16T20:24:02.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore": "bluestore", 2020-11-16T20:24:02.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_fuse": "false", 2020-11-16T20:24:02.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_tracing": "false", 2020-11-16T20:24:02.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_complaint_time": "30.000000", 2020-11-16T20:24:02.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_duration": "600", 2020-11-16T20:24:02.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_size": "20", 2020-11-16T20:24:02.568 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_slow_op_size": "20", 2020-11-16T20:24:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2020-11-16T20:24:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_log_threshold": "5", 2020-11-16T20:24:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards": "0", 2020-11-16T20:24:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards_hdd": "5", 2020-11-16T20:24:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards_ssd": "8", 2020-11-16T20:24:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard": "0", 2020-11-16T20:24:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2020-11-16T20:24:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2020-11-16T20:24:02.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2020-11-16T20:24:02.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_pq_min_cost": "65536", 2020-11-16T20:24:02.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_queue": "wpq", 2020-11-16T20:24:02.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_queue_cut_off": "high", 2020-11-16T20:24:02.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_thread_suicide_timeout": "150", 2020-11-16T20:24:02.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_thread_timeout": "15", 2020-11-16T20:24:02.570 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_open_classes_on_start": "true", 2020-11-16T20:24:02.571 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_os_flags": "0", 2020-11-16T20:24:02.571 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_peering_op_priority": "255", 2020-11-16T20:24:02.571 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_delete_cost": "1048576", 2020-11-16T20:24:02.571 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_delete_priority": "5", 2020-11-16T20:24:02.571 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2020-11-16T20:24:02.571 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2020-11-16T20:24:02.571 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_dups_tracked": "3000", 2020-11-16T20:24:02.571 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_trim_max": "10000", 2020-11-16T20:24:02.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_trim_min": "100", 2020-11-16T20:24:02.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2020-11-16T20:24:02.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_object_context_cache_count": "64", 2020-11-16T20:24:02.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_stat_report_interval_max": "500", 2020-11-16T20:24:02.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2020-11-16T20:24:02.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_min_evict_age": "0", 2020-11-16T20:24:02.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_min_flush_age": "0", 2020-11-16T20:24:02.572 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2020-11-16T20:24:02.573 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2020-11-16T20:24:02.573 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2020-11-16T20:24:02.573 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_crush_rule": "-1", 2020-11-16T20:24:02.573 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_ec_fast_read": "false", 2020-11-16T20:24:02.573 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2020-11-16T20:24:02.573 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_hashpspool": "true", 2020-11-16T20:24:02.573 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nodelete": "false", 2020-11-16T20:24:02.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nopgchange": "false", 2020-11-16T20:24:02.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nosizechange": "false", 2020-11-16T20:24:02.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flags": "0", 2020-11-16T20:24:02.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2020-11-16T20:24:02.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_min_size": "0", 2020-11-16T20:24:02.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2020-11-16T20:24:02.574 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pg_num": "32", 2020-11-16T20:24:02.575 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pgp_num": "0", 2020-11-16T20:24:02.575 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2020-11-16T20:24:02.575 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_size": "2", 2020-11-16T20:24:02.575 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_type": "replicated", 2020-11-16T20:24:02.575 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2020-11-16T20:24:02.575 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_use_gmt_hitset": "true", 2020-11-16T20:24:02.575 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_push_per_object_cost": "1000", 2020-11-16T20:24:02.575 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_read_ec_check_for_errors": "false", 2020-11-16T20:24:02.576 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recover_clone_overlap": "true", 2020-11-16T20:24:02.576 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recover_clone_overlap_limit": "10", 2020-11-16T20:24:02.576 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_cost": "20971520", 2020-11-16T20:24:02.576 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_delay_start": "0.000000", 2020-11-16T20:24:02.576 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active": "0", 2020-11-16T20:24:02.576 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active_hdd": "3", 2020-11-16T20:24:02.576 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active_ssd": "10", 2020-11-16T20:24:02.576 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_chunk": "8388608", 2020-11-16T20:24:02.577 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2020-11-16T20:24:02.577 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_single_start": "1", 2020-11-16T20:24:02.577 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_op_priority": "3", 2020-11-16T20:24:02.577 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_op_warn_multiple": "16", 2020-11-16T20:24:02.577 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_priority": "5", 2020-11-16T20:24:02.577 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_retry_interval": "30.000000", 2020-11-16T20:24:02.577 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep": "0.000000", 2020-11-16T20:24:02.578 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_hdd": "0.100000", 2020-11-16T20:24:02.578 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2020-11-16T20:24:02.578 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_ssd": "0.000000", 2020-11-16T20:24:02.578 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_repair_during_recovery": "false", 2020-11-16T20:24:02.578 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_requested_scrub_priority": "120", 2020-11-16T20:24:02.578 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_rollback_to_cluster_snap": "", 2020-11-16T20:24:02.578 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_auto_repair": "false", 2020-11-16T20:24:02.579 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_auto_repair_num_errors": "5", 2020-11-16T20:24:02.579 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_backoff_ratio": "0.660000", 2020-11-16T20:24:02.579 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_begin_hour": "0", 2020-11-16T20:24:02.579 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_begin_week_day": "0", 2020-11-16T20:24:02.579 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_chunk_max": "25", 2020-11-16T20:24:02.579 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_chunk_min": "5", 2020-11-16T20:24:02.579 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_cost": "52428800", 2020-11-16T20:24:02.579 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_during_recovery": "false", 2020-11-16T20:24:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_end_hour": "0", 2020-11-16T20:24:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_end_week_day": "0", 2020-11-16T20:24:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_extended_sleep": "0.000000", 2020-11-16T20:24:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2020-11-16T20:24:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_invalid_stats": "true", 2020-11-16T20:24:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_load_threshold": "0.500000", 2020-11-16T20:24:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_max_interval": "604800.000000", 2020-11-16T20:24:02.580 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_max_preemptions": "5", 2020-11-16T20:24:02.581 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_min_interval": "86400.000000", 2020-11-16T20:24:02.581 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_priority": "5", 2020-11-16T20:24:02.581 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_sleep": "0.000000", 2020-11-16T20:24:02.581 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_shutdown_pgref_assert": "false", 2020-11-16T20:24:02.581 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_skip_data_digest": "false", 2020-11-16T20:24:02.581 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_smart_report_timeout": "5", 2020-11-16T20:24:02.581 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_cost": "1048576", 2020-11-16T20:24:02.582 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_priority": "5", 2020-11-16T20:24:02.582 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep": "0.000000", 2020-11-16T20:24:02.582 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2020-11-16T20:24:02.582 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2020-11-16T20:24:02.582 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2020-11-16T20:24:02.582 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_stats_ack_timeout_decay": "0.900000", 2020-11-16T20:24:02.582 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_stats_ack_timeout_factor": "2.000000", 2020-11-16T20:24:02.582 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2020-11-16T20:24:02.583 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_target_transaction_size": "30", 2020-11-16T20:24:02.583 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_count": "4", 2020-11-16T20:24:02.583 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2020-11-16T20:24:02.583 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2020-11-16T20:24:02.583 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2020-11-16T20:24:02.583 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2020-11-16T20:24:02.583 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2020-11-16T20:24:02.584 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2020-11-16T20:24:02.584 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_mode": "writeback", 2020-11-16T20:24:02.584 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2020-11-16T20:24:02.584 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_promote_max_objects_sec": "25", 2020-11-16T20:24:02.584 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tracing": "false", 2020-11-16T20:24:02.584 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_use_stale_snap": "false", 2020-11-16T20:24:02.584 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2020-11-16T20:24:02.584 INFO:teuthology.orchestra.run.smithi092.stdout: "osdc_blkin_trace_all": "false", 2020-11-16T20:24:02.585 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_kill_at": "0", 2020-11-16T20:24:02.585 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_max_join_drift": "10", 2020-11-16T20:24:02.585 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_min": "500", 2020-11-16T20:24:02.585 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_min_wait": "0.050000", 2020-11-16T20:24:02.585 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_propose_interval": "1.000000", 2020-11-16T20:24:02.585 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_max": "500", 2020-11-16T20:24:02.585 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_min": "10", 2020-11-16T20:24:02.585 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_stash_full_interval": "25", 2020-11-16T20:24:02.586 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_trim_max": "500", 2020-11-16T20:24:02.586 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_trim_min": "250", 2020-11-16T20:24:02.586 INFO:teuthology.orchestra.run.smithi092.stdout: "perf": "true", 2020-11-16T20:24:02.586 INFO:teuthology.orchestra.run.smithi092.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2020-11-16T20:24:02.586 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin_crypto_accelerator": "crypto_isal", 2020-11-16T20:24:02.586 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin_dir": "/usr/lib/ceph", 2020-11-16T20:24:02.586 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "-", 2020-11-16T20:24:02.587 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrv": "", 2020-11-16T20:24:02.587 INFO:teuthology.orchestra.run.smithi092.stdout: "public_bind_addr": "-", 2020-11-16T20:24:02.587 INFO:teuthology.orchestra.run.smithi092.stdout: "public_network": "", 2020-11-16T20:24:02.587 INFO:teuthology.orchestra.run.smithi092.stdout: "public_network_interface": "", 2020-11-16T20:24:02.587 INFO:teuthology.orchestra.run.smithi092.stdout: "qat_compressor_enabled": "false", 2020-11-16T20:24:02.587 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_mon_op_timeout": "0", 2020-11-16T20:24:02.587 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_osd_op_timeout": "0", 2020-11-16T20:24:02.587 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_tracing": "false", 2020-11-16T20:24:02.588 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_atime_update_interval": "60", 2020-11-16T20:24:02.588 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2020-11-16T20:24:02.588 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_balance_parent_reads": "false", 2020-11-16T20:24:02.588 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_balance_snap_reads": "false", 2020-11-16T20:24:02.588 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blkin_trace_all": "false", 2020-11-16T20:24:02.588 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blocklist_expire_seconds": "0", 2020-11-16T20:24:02.588 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blocklist_on_break_lock": "true", 2020-11-16T20:24:02.589 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache": "true", 2020-11-16T20:24:02.589 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_block_writes_upfront": "false", 2020-11-16T20:24:02.589 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty": "25165824", 2020-11-16T20:24:02.589 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty_age": "1.000000", 2020-11-16T20:24:02.589 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty_object": "0", 2020-11-16T20:24:02.589 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_policy": "writearound", 2020-11-16T20:24:02.589 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_size": "33554432", 2020-11-16T20:24:02.589 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_target_dirty": "16777216", 2020-11-16T20:24:02.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_writethrough_until_flush": "true", 2020-11-16T20:24:02.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_clone_copy_on_read": "false", 2020-11-16T20:24:02.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_compression_hint": "none", 2020-11-16T20:24:02.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_concurrent_management_ops": "10", 2020-11-16T20:24:02.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_config_pool_override_update_timestamp": "0", 2020-11-16T20:24:02.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_clone_format": "auto", 2020-11-16T20:24:02.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_data_pool": "", 2020-11-16T20:24:02.590 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_features": "61", 2020-11-16T20:24:02.591 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_format": "2", 2020-11-16T20:24:02.591 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_map_options": "", 2020-11-16T20:24:02.591 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_order": "22", 2020-11-16T20:24:02.591 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_pool": "rbd", 2020-11-16T20:24:02.591 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2020-11-16T20:24:02.591 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_stripe_count": "0", 2020-11-16T20:24:02.591 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_stripe_unit": "0", 2020-11-16T20:24:02.592 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_disable_zero_copy_writes": "true", 2020-11-16T20:24:02.592 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_discard_granularity_bytes": "65536", 2020-11-16T20:24:02.592 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_discard_on_zeroed_write_same": "true", 2020-11-16T20:24:02.592 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_enable_alloc_hint": "true", 2020-11-16T20:24:02.592 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2020-11-16T20:24:02.592 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_io_scheduler": "simple", 2020-11-16T20:24:02.592 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2020-11-16T20:24:02.592 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_commit_age": "5.000000", 2020-11-16T20:24:02.593 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2020-11-16T20:24:02.593 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_max_payload_bytes": "16384", 2020-11-16T20:24:02.593 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_age": "0.000000", 2020-11-16T20:24:02.593 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_bytes": "1048576", 2020-11-16T20:24:02.593 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_interval": "0", 2020-11-16T20:24:02.593 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2020-11-16T20:24:02.593 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2020-11-16T20:24:02.593 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_order": "24", 2020-11-16T20:24:02.594 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_pool": "", 2020-11-16T20:24:02.594 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_splay_width": "4", 2020-11-16T20:24:02.594 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_localize_parent_reads": "false", 2020-11-16T20:24:02.594 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_localize_snap_reads": "false", 2020-11-16T20:24:02.594 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2020-11-16T20:24:02.594 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2020-11-16T20:24:02.594 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2020-11-16T20:24:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2020-11-16T20:24:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2020-11-16T20:24:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2020-11-16T20:24:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_type": "simple", 2020-11-16T20:24:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2020-11-16T20:24:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_state_check_interval": "30", 2020-11-16T20:24:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2020-11-16T20:24:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2020-11-16T20:24:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2020-11-16T20:24:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2020-11-16T20:24:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2020-11-16T20:24:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_autotune": "true", 2020-11-16T20:24:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_base": "805306368", 2020-11-16T20:24:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2020-11-16T20:24:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_min": "134217728", 2020-11-16T20:24:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2020-11-16T20:24:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2020-11-16T20:24:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_target": "4294967296", 2020-11-16T20:24:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_perf_stats_prio": "5", 2020-11-16T20:24:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2020-11-16T20:24:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2020-11-16T20:24:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_delete_delay": "0", 2020-11-16T20:24:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_max_mirroring_snapshots": "3", 2020-11-16T20:24:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_replay_delay": "0", 2020-11-16T20:24:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2020-11-16T20:24:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2020-11-16T20:24:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_to_trash_on_remove": "false", 2020-11-16T20:24:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2020-11-16T20:24:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mtime_update_interval": "60", 2020-11-16T20:24:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_non_blocking_aio": "true", 2020-11-16T20:24:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_op_thread_timeout": "60", 2020-11-16T20:24:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_op_threads": "1", 2020-11-16T20:24:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_parent_cache_enabled": "false", 2020-11-16T20:24:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_plugins": "", 2020-11-16T20:24:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_burst": "0", 2020-11-16T20:24:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_burst_seconds": "1", 2020-11-16T20:24:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_limit": "0", 2020-11-16T20:24:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_burst": "0", 2020-11-16T20:24:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_burst_seconds": "1", 2020-11-16T20:24:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_limit": "0", 2020-11-16T20:24:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_burst": "0", 2020-11-16T20:24:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2020-11-16T20:24:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_limit": "0", 2020-11-16T20:24:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_burst": "0", 2020-11-16T20:24:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2020-11-16T20:24:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_limit": "0", 2020-11-16T20:24:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_schedule_tick_min": "50", 2020-11-16T20:24:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_burst": "0", 2020-11-16T20:24:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2020-11-16T20:24:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_limit": "0", 2020-11-16T20:24:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_burst": "0", 2020-11-16T20:24:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2020-11-16T20:24:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_limit": "0", 2020-11-16T20:24:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_quiesce_notification_attempts": "10", 2020-11-16T20:24:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_read_from_replica_policy": "default", 2020-11-16T20:24:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2020-11-16T20:24:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_max_bytes": "524288", 2020-11-16T20:24:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_trigger_requests": "10", 2020-11-16T20:24:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_request_timed_out_seconds": "30", 2020-11-16T20:24:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_rwl_enabled": "false", 2020-11-16T20:24:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_rwl_log_periodic_stats": "false", 2020-11-16T20:24:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_rwl_path": "/tmp", 2020-11-16T20:24:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_rwl_size": "1073741824", 2020-11-16T20:24:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_skip_partial_discard": "true", 2020-11-16T20:24:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2020-11-16T20:24:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_tracing": "false", 2020-11-16T20:24:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_validate_names": "true", 2020-11-16T20:24:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_validate_pool": "true", 2020-11-16T20:24:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "restapi_base_url": "", 2020-11-16T20:24:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "restapi_log_level": "", 2020-11-16T20:24:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_acl_grants_max_num": "100", 2020-11-16T20:24:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_admin_entry": "admin", 2020-11-16T20:24:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_barbican_url": "", 2020-11-16T20:24:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2020-11-16T20:24:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_default_quota_max_size": "-1", 2020-11-16T20:24:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_index_max_aio": "128", 2020-11-16T20:24:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_cache_size": "10000", 2020-11-16T20:24:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_soft_threshold": "0.950000", 2020-11-16T20:24:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_ttl": "600", 2020-11-16T20:24:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_enabled": "true", 2020-11-16T20:24:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_expiry_interval": "900", 2020-11-16T20:24:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_lru_size": "10000", 2020-11-16T20:24:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_content_length_compat": "false", 2020-11-16T20:24:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_copy_obj_progress": "true", 2020-11-16T20:24:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2020-11-16T20:24:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cors_rules_max_num": "100", 2020-11-16T20:24:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cross_domain_policy": "", 2020-11-16T20:24:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_default_encryption_key": "", 2020-11-16T20:24:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_require_ssl": "true", 2020-11-16T20:24:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2020-11-16T20:24:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2020-11-16T20:24:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_suppress_logs": "true", 2020-11-16T20:24:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_addr": "", 2020-11-16T20:24:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_auth": "token", 2020-11-16T20:24:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_namespace": "", 2020-11-16T20:24:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_prefix": "", 2020-11-16T20:24:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_secret_engine": "transit", 2020-11-16T20:24:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_token_file": "", 2020-11-16T20:24:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_low_speed_limit": "1024", 2020-11-16T20:24:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_low_speed_time": "300", 2020-11-16T20:24:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_wait_timeout_ms": "1000", 2020-11-16T20:24:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2020-11-16T20:24:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_backing": "auto", 2020-11-16T20:24:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_changes_size": "1000", 2020-11-16T20:24:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_num_shards": "128", 2020-11-16T20:24:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_obj_prefix": "data_log", 2020-11-16T20:24:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_window": "30", 2020-11-16T20:24:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_notify_interval_msec": "200", 2020-11-16T20:24:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_realm_info_oid": "default.realm", 2020-11-16T20:24:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_region_info_oid": "default.region", 2020-11-16T20:24:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_zone_info_oid": "default.zone", 2020-11-16T20:24:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2020-11-16T20:24:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_defer_to_bucket_acls": "", 2020-11-16T20:24:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_delete_multi_obj_max_num": "1000", 2020-11-16T20:24:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_lim": "0.000000", 2020-11-16T20:24:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_res": "100.000000", 2020-11-16T20:24:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2020-11-16T20:24:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_lim": "0.000000", 2020-11-16T20:24:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_res": "200.000000", 2020-11-16T20:24:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2020-11-16T20:24:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_lim": "0.000000", 2020-11-16T20:24:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_res": "500.000000", 2020-11-16T20:24:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_wgt": "500.000000", 2020-11-16T20:24:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2020-11-16T20:24:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_res": "500.000000", 2020-11-16T20:24:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2020-11-16T20:24:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dns_name": "", 2020-11-16T20:24:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dns_s3website_name": "", 2020-11-16T20:24:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dynamic_resharding": "true", 2020-11-16T20:24:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, pubsub", 2020-11-16T20:24:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_gc_threads": "true", 2020-11-16T20:24:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_lc_threads": "true", 2020-11-16T20:24:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_ops_log": "false", 2020-11-16T20:24:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_quota_threads": "true", 2020-11-16T20:24:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_static_website": "false", 2020-11-16T20:24:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_usage_log": "false", 2020-11-16T20:24:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enforce_swift_acls": "true", 2020-11-16T20:24:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_exit_timeout_secs": "120", 2020-11-16T20:24:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_expose_bucket": "false", 2020-11-16T20:24:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_extended_http_attrs": "", 2020-11-16T20:24:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_fcgi_socket_backlog": "1024", 2020-11-16T20:24:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2020-11-16T20:24:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_frontends": "beast port=7480", 2020-11-16T20:24:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_concurrent_io": "10", 2020-11-16T20:24:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_deferred": "50", 2020-11-16T20:24:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2020-11-16T20:24:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_objs": "32", 2020-11-16T20:24:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_queue_size": "134213632", 2020-11-16T20:24:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_trim_chunk": "16", 2020-11-16T20:24:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_obj_min_wait": "7200", 2020-11-16T20:24:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_processor_max_time": "3600", 2020-11-16T20:24:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_processor_period": "3600", 2020-11-16T20:24:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_get_obj_max_req_size": "4194304", 2020-11-16T20:24:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_get_obj_window_size": "16777216", 2020-11-16T20:24:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_healthcheck_disabling_path": "", 2020-11-16T20:24:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_host": "", 2020-11-16T20:24:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ignore_get_invalid_range": "false", 2020-11-16T20:24:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_init_timeout": "300", 2020-11-16T20:24:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2020-11-16T20:24:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_admin_roles": "", 2020-11-16T20:24:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2020-11-16T20:24:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_domain": "", 2020-11-16T20:24:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_password": "", 2020-11-16T20:24:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_password_path": "", 2020-11-16T20:24:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_project": "", 2020-11-16T20:24:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_tenant": "", 2020-11-16T20:24:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_token": "", 2020-11-16T20:24:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_token_path": "", 2020-11-16T20:24:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_user": "", 2020-11-16T20:24:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_api_version": "2", 2020-11-16T20:24:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_domain": "", 2020-11-16T20:24:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_password": "", 2020-11-16T20:24:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_project": "", 2020-11-16T20:24:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_tenant": "", 2020-11-16T20:24:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_user": "", 2020-11-16T20:24:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_implicit_tenants": "false", 2020-11-16T20:24:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_token_cache_size": "10000", 2020-11-16T20:24:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_url": "", 2020-11-16T20:24:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_verify_ssl": "true", 2020-11-16T20:24:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_debug_interval": "-1", 2020-11-16T20:24:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_lock_max_time": "90", 2020-11-16T20:24:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_objs": "32", 2020-11-16T20:24:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_rules": "1000", 2020-11-16T20:24:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_worker": "3", 2020-11-16T20:24:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_wp_worker": "3", 2020-11-16T20:24:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_thread_delay": "0", 2020-11-16T20:24:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2020-11-16T20:24:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_dnattr": "uid", 2020-11-16T20:24:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2020-11-16T20:24:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_searchfilter": "", 2020-11-16T20:24:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2020-11-16T20:24:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_uri": "ldaps://", 2020-11-16T20:24:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2020-11-16T20:24:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_list_bucket_min_readahead": "1000", 2020-11-16T20:24:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_list_buckets_max_chunk": "1000", 2020-11-16T20:24:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_http_headers": "", 2020-11-16T20:24:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_nonexistent_bucket": "false", 2020-11-16T20:24:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2020-11-16T20:24:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_object_name_utc": "false", 2020-11-16T20:24:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attr_name_len": "0", 2020-11-16T20:24:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attr_size": "0", 2020-11-16T20:24:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attrs_num_in_req": "0", 2020-11-16T20:24:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_chunk_size": "4194304", 2020-11-16T20:24:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_concurrent_requests": "1024", 2020-11-16T20:24:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_dynamic_shards": "1999", 2020-11-16T20:24:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_listing_results": "1000", 2020-11-16T20:24:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_notify_retries": "3", 2020-11-16T20:24:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_objs_per_shard": "100000", 2020-11-16T20:24:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_put_param_size": "1048576", 2020-11-16T20:24:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_put_size": "5368709120", 2020-11-16T20:24:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_slo_entries": "1000", 2020-11-16T20:24:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_md_log_max_shards": "64", 2020-11-16T20:24:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_md_notify_interval_msec": "200", 2020-11-16T20:24:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_mime_types_file": "/etc/mime.types", 2020-11-16T20:24:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_mp_lock_max_time": "600", 2020-11-16T20:24:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multipart_min_part_size": "5242880", 2020-11-16T20:24:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multipart_part_upload_limit": "10000", 2020-11-16T20:24:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_fhcache_partitions": "3", 2020-11-16T20:24:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_fhcache_size": "2017", 2020-11-16T20:24:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2020-11-16T20:24:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_lru_lanes": "5", 2020-11-16T20:24:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_max_gc": "300", 2020-11-16T20:24:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_namespace_expire_secs": "300", 2020-11-16T20:24:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_gc_threads": "false", 2020-11-16T20:24:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_lc_threads": "false", 2020-11-16T20:24:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_quota_threads": "false", 2020-11-16T20:24:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_sync_thread": "false", 2020-11-16T20:24:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_s3_fast_attrs": "false", 2020-11-16T20:24:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_write_completion_interval_s": "10", 2020-11-16T20:24:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_async_rados_threads": "32", 2020-11-16T20:24:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_control_oids": "8", 2020-11-16T20:24:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_rados_handles": "1", 2020-11-16T20:24:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_numa_node": "-1", 2020-11-16T20:24:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_obj_stripe_size": "4194304", 2020-11-16T20:24:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_obj_tombstone_cache_size": "1000", 2020-11-16T20:24:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_chunk_size": "100", 2020-11-16T20:24:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_gc_interval": "600", 2020-11-16T20:24:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_hints_num_shards": "127", 2020-11-16T20:24:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_olh_pending_timeout_sec": "3600", 2020-11-16T20:24:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_thread_suicide_timeout": "0", 2020-11-16T20:24:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_thread_timeout": "600", 2020-11-16T20:24:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_tracing": "false", 2020-11-16T20:24:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_token": "", 2020-11-16T20:24:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_url": "", 2020-11-16T20:24:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_verify_ssl": "true", 2020-11-16T20:24:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_data_backlog": "5242880", 2020-11-16T20:24:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_rados": "true", 2020-11-16T20:24:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_socket_path": "", 2020-11-16T20:24:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_override_bucket_index_max_shards": "0", 2020-11-16T20:24:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2020-11-16T20:24:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_push_interval": "2.000000", 2020-11-16T20:24:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_push_interval_max": "30.000000", 2020-11-16T20:24:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_root_pool": ".rgw.root", 2020-11-16T20:24:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_port": "", 2020-11-16T20:24:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_print_continue": "true", 2020-11-16T20:24:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_print_prohibited_content_length": "false", 2020-11-16T20:24:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_put_obj_max_window_size": "67108864", 2020-11-16T20:24:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_put_obj_min_window_size": "16777216", 2020-11-16T20:24:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2020-11-16T20:24:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_pg_num_min": "8", 2020-11-16T20:24:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_recovery_priority": "5", 2020-11-16T20:24:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_tracing": "false", 2020-11-16T20:24:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm": "", 2020-11-16T20:24:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm_root_pool": ".rgw.root", 2020-11-16T20:24:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_region": "", 2020-11-16T20:24:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_region_root_pool": ".rgw.root", 2020-11-16T20:24:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_relaxed_region_enforcement": "false", 2020-11-16T20:24:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_relaxed_s3_bucket_names": "false", 2020-11-16T20:24:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2020-11-16T20:24:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_request_uri": "", 2020-11-16T20:24:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_batch_size": "64", 2020-11-16T20:24:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_bucket_lock_duration": "360", 2020-11-16T20:24:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_max_aio": "128", 2020-11-16T20:24:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_num_logs": "16", 2020-11-16T20:24:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_thread_interval": "600", 2020-11-16T20:24:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_resolve_cname": "false", 2020-11-16T20:24:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rest_getusage_op_compat": "false", 2020-11-16T20:24:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_run_sync_thread": "true", 2020-11-16T20:24:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_order": "sts, external, local", 2020-11-16T20:24:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_keystone": "false", 2020-11-16T20:24:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_ldap": "false", 2020-11-16T20:24:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_rados": "true", 2020-11-16T20:24:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_sts": "false", 2020-11-16T20:24:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_success_create_obj_status": "0", 2020-11-16T20:24:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_safe_max_objects_per_shard": "102400", 2020-11-16T20:24:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_scheduler_type": "throttler", 2020-11-16T20:24:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_script_uri": "", 2020-11-16T20:24:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_service_provider_name": "", 2020-11-16T20:24:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_shard_warning_threshold": "90.000000", 2020-11-16T20:24:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_socket_path": "", 2020-11-16T20:24:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_client_id": "", 2020-11-16T20:24:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_client_secret": "", 2020-11-16T20:24:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_entry": "sts", 2020-11-16T20:24:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_key": "sts", 2020-11-16T20:24:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_max_session_duration": "43200", 2020-11-16T20:24:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_min_session_duration": "900", 2020-11-16T20:24:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_token_introspection_url": "", 2020-11-16T20:24:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_account_in_url": "false", 2020-11-16T20:24:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_auth_entry": "auth", 2020-11-16T20:24:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_auth_url": "", 2020-11-16T20:24:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_custom_header": "", 2020-11-16T20:24:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_enforce_content_length": "false", 2020-11-16T20:24:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_need_stats": "true", 2020-11-16T20:24:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_tenant_name": "", 2020-11-16T20:24:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_token_expiration": "86400", 2020-11-16T20:24:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_url": "", 2020-11-16T20:24:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_url_prefix": "swift", 2020-11-16T20:24:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_versioning_enabled": "false", 2020-11-16T20:24:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2020-11-16T20:24:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_lease_period": "120", 2020-11-16T20:24:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2020-11-16T20:24:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_interval": "1200", 2020-11-16T20:24:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_max_buckets": "16", 2020-11-16T20:24:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2020-11-16T20:24:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2020-11-16T20:24:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_obj_etag_verify": "false", 2020-11-16T20:24:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_history_size": "4096", 2020-11-16T20:24:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_per_node_log_size": "32", 2020-11-16T20:24:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2020-11-16T20:24:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_thread_pool_size": "512", 2020-11-16T20:24:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_comment": "", 2020-11-16T20:24:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_createby": "", 2020-11-16T20:24:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_encoding": "", 2020-11-16T20:24:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_flag": "false", 2020-11-16T20:24:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_origin": "", 2020-11-16T20:24:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_sha_unit": "524288", 2020-11-16T20:24:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_tracker": "", 2020-11-16T20:24:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_trust_forwarded_https": "false", 2020-11-16T20:24:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_log_flush_threshold": "1024", 2020-11-16T20:24:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_log_tick_interval": "30", 2020-11-16T20:24:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_max_shards": "32", 2020-11-16T20:24:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_max_user_shards": "1", 2020-11-16T20:24:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_use_opa_authz": "false", 2020-11-16T20:24:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_default_quota_max_objects": "-1", 2020-11-16T20:24:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_default_quota_max_size": "-1", 2020-11-16T20:24:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_max_buckets": "1000", 2020-11-16T20:24:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2020-11-16T20:24:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_idle_users": "false", 2020-11-16T20:24:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_interval": "86400", 2020-11-16T20:24:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_wait_time": "86400", 2020-11-16T20:24:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_unique_email": "true", 2020-11-16T20:24:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_verify_ssl": "true", 2020-11-16T20:24:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_website_routing_rules_max_num": "50", 2020-11-16T20:24:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone": "", 2020-11-16T20:24:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone_root_pool": ".rgw.root", 2020-11-16T20:24:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup": "", 2020-11-16T20:24:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2020-11-16T20:24:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_block_size": "4096", 2020-11-16T20:24:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_bloom_bits_per_key": "20", 2020-11-16T20:24:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2020-11-16T20:24:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "true", 2020-11-16T20:24:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_row_ratio": "0.000000", 2020-11-16T20:24:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_shard_bits": "4", 2020-11-16T20:24:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_size": "536870912", 2020-11-16T20:24:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_type": "binned_lru", 2020-11-16T20:24:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_compaction_stats": "false", 2020-11-16T20:24:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_extended_stats": "false", 2020-11-16T20:24:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_memory_stats": "false", 2020-11-16T20:24:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_delete_range_threshold": "1048576", 2020-11-16T20:24:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_index_type": "binary_search", 2020-11-16T20:24:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_log_to_ceph_log": "true", 2020-11-16T20:24:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_metadata_block_size": "4096", 2020-11-16T20:24:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_partition_filters": "false", 2020-11-16T20:24:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_perf": "false", 2020-11-16T20:24:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2020-11-16T20:24:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "rotating_keys_bootstrap_timeout": "30", 2020-11-16T20:24:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "rotating_keys_renewal_timeout": "10", 2020-11-16T20:24:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "run_dir": "/var/run/ceph", 2020-11-16T20:24:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "setgroup": "", 2020-11-16T20:24:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "setuser": "", 2020-11-16T20:24:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "setuser_match_path": "", 2020-11-16T20:24:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_misplaced_ratio": "0.050000", 2020-11-16T20:24:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "thp": "false", 2020-11-16T20:24:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "threadpool_default_timeout": "60", 2020-11-16T20:24:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "threadpool_empty_queue_max_wait": "2", 2020-11-16T20:24:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "throttler_perf_counter": "true" 2020-11-16T20:24:02.645 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:02.647 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2020-11-16T20:24:02.807 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b: {} 2020-11-16T20:24:02.827 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a: {} 2020-11-16T20:24:02.846 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c: {} 2020-11-16T20:24:02.847 INFO:teuthology.orchestra.run.smithi092.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-11-16T20:24:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-11-16T20:24:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-11-16T20:24:02.861 INFO:teuthology.orchestra.run.smithi013:> type ceph-objectstore-tool 2020-11-16T20:24:02.867 INFO:teuthology.orchestra.run.smithi013.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2020-11-16T20:24:02.868 INFO:teuthology.orchestra.run.smithi092:> type ceph-objectstore-tool 2020-11-16T20:24:02.872 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2020-11-16T20:24:02.873 INFO:teuthology.run_tasks:Running task rados... 2020-11-16T20:24:02.883 INFO:tasks.rados:Beginning rados... 2020-11-16T20:24:02.884 DEBUG:teuthology.run_tasks:Unwinding manager rados 2020-11-16T20:24:02.921 INFO:tasks.rados:joining rados 2020-11-16T20:24:03.049 INFO:tasks.thrashosds.thrasher:starting do_thrash 2020-11-16T20:24:03.050 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-11-16T20:24:03.050 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T20:24:03.051 INFO:tasks.thrashosds.thrasher:Growing pool device_health_metrics 2020-11-16T20:24:03.052 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T20:24:03.052 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2020-11-16T20:24:03.053 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2020-11-16T20:24:03.053 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T20:24:03.054 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2020-11-16T20:24:03.054 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T20:24:03.055 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2020-11-16T20:24:03.055 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T20:24:03.056 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2020-11-16T20:24:03.058 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:03.050+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:03.156 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:03.162+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:03.188 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:03.188 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:03.188 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:03.189 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:03.201 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T20:24:03.229 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:03.268 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:03.269 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:03.274+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:03.302 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:03.327 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:03.343 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:03.343 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:03.354 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T20:24:03.357 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:03.354+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:03.358 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:24:03.388 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2020-11-16T20:24:03.390 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:03.401 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2020-11-16T20:24:01.618373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39808,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38336,"kb_avail":754934912,"statfs":{"total":773094113280,"available":773053349888,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39256064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003194"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.606444+0000","last_change":"2020-11-16T20:23:44.605343+0000","last_active":"2020-11-16T20:23:44.606444+0000","last_peered":"2020-11-16T20:23:44.606444+0000","last_clean":"2020-11-16T20:23:44.606444+0000","last_became_active":"2020-11-16T20:23:44.604919+0000","last_became_peered":"2020-11-16T20:23:44.604919+0000","last_unstale":"2020-11-16T20:23:44.606444+0000","last_undegraded":"2020-11-16T20:23:44.606444+0000","last_fullsized":"2020-11-16T20:23:44.606444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600239+0000","last_change":"2020-11-16T20:23:44.598654+0000","last_active":"2020-11-16T20:23:44.600239+0000","last_peered":"2020-11-16T20:23:44.600239+0000","last_clean":"2020-11-16T20:23:44.600239+0000","last_became_active":"2020-11-16T20:23:44.597867+0000","last_became_peered":"2020-11-16T20:23:44.597867+0000","last_unstale":"2020-11-16T20:23:44.600239+0000","last_undegraded":"2020-11-16T20:23:44.600239+0000","last_fullsized":"2020-11-16T20:23:44.600239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502968+0000","last_change":"2020-11-16T20:23:44.501820+0000","last_active":"2020-11-16T20:23:44.502968+0000","last_peered":"2020-11-16T20:23:44.502968+0000","last_clean":"2020-11-16T20:23:44.502968+0000","last_became_active":"2020-11-16T20:23:44.501424+0000","last_became_peered":"2020-11-16T20:23:44.501424+0000","last_unstale":"2020-11-16T20:23:44.502968+0000","last_undegraded":"2020-11-16T20:23:44.502968+0000","last_fullsized":"2020-11-16T20:23:44.502968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.504330+0000","last_change":"2020-11-16T20:23:44.503055+0000","last_active":"2020-11-16T20:23:44.504330+0000","last_peered":"2020-11-16T20:23:44.504330+0000","last_clean":"2020-11-16T20:23:44.504330+0000","last_became_active":"2020-11-16T20:23:44.502588+0000","last_became_peered":"2020-11-16T20:23:44.502588+0000","last_unstale":"2020-11-16T20:23:44.504330+0000","last_undegraded":"2020-11-16T20:23:44.504330+0000","last_fullsized":"2020-11-16T20:23:44.504330+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.493541+0000","last_change":"2020-11-16T20:23:39.780867+0000","last_active":"2020-11-16T20:23:44.493541+0000","last_peered":"2020-11-16T20:23:44.493541+0000","last_clean":"2020-11-16T20:23:44.493541+0000","last_became_active":"2020-11-16T20:23:39.779329+0000","last_became_peered":"2020-11-16T20:23:39.779329+0000","last_unstale":"2020-11-16T20:23:44.493541+0000","last_undegraded":"2020-11-16T20:23:44.493541+0000","last_fullsized":"2020-11-16T20:23:44.493541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:23:36.602638+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.502939+0000","last_change":"2020-11-16T20:23:44.501339+0000","last_active":"2020-11-16T20:23:44.502939+0000","last_peered":"2020-11-16T20:23:44.502939+0000","last_clean":"2020-11-16T20:23:44.502939+0000","last_became_active":"2020-11-16T20:23:44.500782+0000","last_became_peered":"2020-11-16T20:23:44.500782+0000","last_unstale":"2020-11-16T20:23:44.502939+0000","last_undegraded":"2020-11-16T20:23:44.502939+0000","last_fullsized":"2020-11-16T20:23:44.502939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.494389+0000","last_change":"2020-11-16T20:23:44.493249+0000","last_active":"2020-11-16T20:23:44.494389+0000","last_peered":"2020-11-16T20:23:44.494389+0000","last_clean":"2020-11-16T20:23:44.494389+0000","last_became_active":"2020-11-16T20:23:44.492834+0000","last_became_peered":"2020-11-16T20:23:44.492834+0000","last_unstale":"2020-11-16T20:23:44.494389+0000","last_undegraded":"2020-11-16T20:23:44.494389+0000","last_fullsized":"2020-11-16T20:23:44.494389+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.607721+0000","last_change":"2020-11-16T20:23:44.606663+0000","last_active":"2020-11-16T20:23:44.607721+0000","last_peered":"2020-11-16T20:23:44.607721+0000","last_clean":"2020-11-16T20:23:44.607721+0000","last_became_active":"2020-11-16T20:23:44.605904+0000","last_became_peered":"2020-11-16T20:23:44.605904+0000","last_unstale":"2020-11-16T20:23:44.607721+0000","last_undegraded":"2020-11-16T20:23:44.607721+0000","last_fullsized":"2020-11-16T20:23:44.607721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2020-11-16T20:23:44.600231+0000","last_change":"2020-11-16T20:23:44.599215+0000","last_active":"2020-11-16T20:23:44.600231+0000","last_peered":"2020-11-16T20:23:44.600231+0000","last_clean":"2020-11-16T20:23:44.600231+0000","last_became_active":"2020-11-16T20:23:44.597937+0000","last_became_peered":"2020-11-16T20:23:44.597937+0000","last_unstale":"2020-11-16T20:23:44.600231+0000","last_undegraded":"2020-11-16T20:23:44.600231+0000","last_fullsized":"2020-11-16T20:23:44.600231+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4920,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94366920,"statfs":{"total":96636764160,"available":96631726080,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.96}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1389999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8969999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7290000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5379999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.734}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3690000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8690000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4119999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4910000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.25}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.669}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8420000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4620000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6880000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.641}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8250000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7650000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0019999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5139999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8939999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0579999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1309999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5760000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9340000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3290000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4620000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9710000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1349999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5579999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.948}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.129}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7770000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4039999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5379999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.472}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5329999999999999}]}]}]}} 2020-11-16T20:24:03.403 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2020-11-16T20:24:03.403 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pg_num 11 2020-11-16T20:24:03.424 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:03.465 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:03.466 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:03.470+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:03.467 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T20:24:03.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T20:24:03.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T20:24:03.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-11-16T20:24:03.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-11-16T20:24:03.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-11-16T20:24:03.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-11-16T20:24:03.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-11-16T20:24:03.492 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:03.493 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:03.503 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2020-11-16T20:24:03.506 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T20:24:03.558 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:03.554+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:03.659 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:03.662+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:03.680 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:03.680 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:03.693 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T20:24:03.780 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:03.786+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:03.856 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:03.856 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:03.861 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:03.866+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:03.866 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T20:24:03.962 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:03.966+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:04.028 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:04.028 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:04.039 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T20:24:04.081 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:04.086+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:04.169 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:04.166+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:04.203 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:04.204 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:04.214 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T20:24:04.270 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:04.266+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:04.333 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:04.334 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:04.344 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T20:24:04.371 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:04.366+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:04.462 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:04.462 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:04.472 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-11-16T20:24:04.472 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:04.470+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 1 pg_num to 11 2020-11-16T20:24:04.518 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set isaprofile crush-failure-domain=osd k=2 m=1 name=isaprofile plugin=isa technique=reed_sol_van 2020-11-16T20:24:04.572 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:04.570+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:04.620 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-11-16T20:24:04.673 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:04.670+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:04.774 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:04.770+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:04.782 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:04.783 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:04.793 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-11-16T20:24:04.875 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:04.870+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:04.904 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:04.904 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:04.915 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-11-16T20:24:04.975 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:04.977+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:05.043 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:05.043 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:05.054 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-11-16T20:24:05.100 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:05.098+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:05.170 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:05.171 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:05.177 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:05.181+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:05.181 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-11-16T20:24:05.277 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:05.274+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:05.344 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:05.344 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:05.354 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-11-16T20:24:05.396 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:05.394+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:05.479 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:05.481+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:05.484 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T20:24:05.515 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T20:24:05.516 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:05.517 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:05.524 INFO:tasks.rados:starting run 0 out of 1 2020-11-16T20:24:05.524 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2020-11-16T20:24:05.525 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure isaprofile 2020-11-16T20:24:05.527 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-11-16T20:24:05.580 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:05.578+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:05.642 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:05.642 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:05.653 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-11-16T20:24:05.655 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:05.693 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:05.695 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:05.694+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:05.719 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:05.746 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:05.777 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:05.782 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:05.785+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:05.807 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:05.817 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:05.817 INFO:teuthology.orchestra.run.smithi013.stdout: "size": 20, 2020-11-16T20:24:05.818 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 600, 2020-11-16T20:24:05.818 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [ 2020-11-16T20:24:05.818 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.818 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_repop(client.4207.0:2 1.0 e14/13 1:08c0073d:::INTEL_SSDPED1D480GA_PHMB7476008K480DGN:head v 14'1, mlcod=0'0)", 2020-11-16T20:24:05.818 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:23:39.775042+0000", 2020-11-16T20:24:05.818 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 26.041465614, 2020-11-16T20:24:05.818 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001769439, 2020-11-16T20:24:05.819 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:05.819 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:05.819 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:05.819 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.819 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:05.819 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.775042+0000", 2020-11-16T20:24:05.819 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:05.819 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.820 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.820 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:05.820 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.775042+0000", 2020-11-16T20:24:05.820 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00013750700000000001 2020-11-16T20:24:05.820 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.820 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.820 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:05.821 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.775180+0000", 2020-11-16T20:24:05.821 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4001e-05 2020-11-16T20:24:05.821 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.821 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.821 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:05.821 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.775194+0000", 2020-11-16T20:24:05.821 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00014091599999999999 2020-11-16T20:24:05.822 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.822 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.822 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:05.822 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.775334+0000", 2020-11-16T20:24:05.822 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.6404000000000001e-05 2020-11-16T20:24:05.822 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.822 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.822 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:05.823 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.775351+0000", 2020-11-16T20:24:05.823 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.0313000000000001e-05 2020-11-16T20:24:05.823 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.823 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.823 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:05.823 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.775441+0000", 2020-11-16T20:24:05.823 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.8927000000000002e-05 2020-11-16T20:24:05.824 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.824 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.824 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:05.824 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.775480+0000", 2020-11-16T20:24:05.824 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0011995210000000001 2020-11-16T20:24:05.824 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.824 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.824 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:05.825 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.776680+0000", 2020-11-16T20:24:05.825 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00013185000000000001 2020-11-16T20:24:05.825 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.825 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.825 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:05.825 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.776812+0000", 2020-11-16T20:24:05.825 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001769439 2020-11-16T20:24:05.826 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:05.826 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:05.826 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:05.826 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.826 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.826 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_repop(client.4207.0:4 1.0 e14/13 1:97e83c24:::INTEL_SSDPEDMD400G4_PHFT620400QM400BGN:head v 14'2, mlcod=0'0)", 2020-11-16T20:24:05.826 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:23:39.908420+0000", 2020-11-16T20:24:05.827 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 25.908088111000001, 2020-11-16T20:24:05.827 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.002204119, 2020-11-16T20:24:05.827 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:05.827 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:05.827 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:05.827 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.827 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:05.827 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.908420+0000", 2020-11-16T20:24:05.828 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:05.828 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.828 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.828 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:05.828 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.908420+0000", 2020-11-16T20:24:05.828 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000105623 2020-11-16T20:24:05.828 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.828 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.829 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:05.829 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.908525+0000", 2020-11-16T20:24:05.829 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.5177e-05 2020-11-16T20:24:05.829 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.829 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.829 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:05.829 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.908550+0000", 2020-11-16T20:24:05.830 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00013881700000000001 2020-11-16T20:24:05.830 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.830 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.830 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:05.830 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.908689+0000", 2020-11-16T20:24:05.830 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4127e-05 2020-11-16T20:24:05.830 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.831 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.831 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:05.831 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.908703+0000", 2020-11-16T20:24:05.831 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000152533 2020-11-16T20:24:05.831 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.831 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.831 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:05.831 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.908856+0000", 2020-11-16T20:24:05.832 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.9660000000000006e-05 2020-11-16T20:24:05.832 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.832 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.832 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:05.832 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.908936+0000", 2020-11-16T20:24:05.832 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001487452 2020-11-16T20:24:05.832 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.833 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.833 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:05.833 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.910423+0000", 2020-11-16T20:24:05.833 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00020073000000000001 2020-11-16T20:24:05.833 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:05.833 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:05.833 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:05.833 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:23:39.910624+0000", 2020-11-16T20:24:05.834 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.002204119 2020-11-16T20:24:05.834 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:05.834 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:05.834 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:05.834 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:05.834 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:05.834 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:05.835 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:05.836 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-11-16T20:24:05.883 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:05.885+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:05.956 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:05.957 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:05.957 INFO:teuthology.orchestra.run.smithi013.stdout: "num_ops": 0 2020-11-16T20:24:05.957 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:05.967 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-11-16T20:24:06.011 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:06.010+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:06.085 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:06.089+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:06.086 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:06.087 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:06.087 INFO:teuthology.orchestra.run.smithi013.stdout: "complaint_time": 30, 2020-11-16T20:24:06.087 INFO:teuthology.orchestra.run.smithi013.stdout: "num_blocked_ops": 0 2020-11-16T20:24:06.087 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:06.097 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-11-16T20:24:06.186 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:06.182+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:06.260 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:06.260 INFO:teuthology.orchestra.run.smithi013.stdout: "size": 20, 2020-11-16T20:24:06.261 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 600, 2020-11-16T20:24:06.261 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [] 2020-11-16T20:24:06.261 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:06.271 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-11-16T20:24:06.312 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:06.310+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:06.388 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:06.388+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:06.433 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:06.433 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:06.443 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-11-16T20:24:06.489 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:06.492+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:06.565 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2020-11-16T20:24:06.567 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:06.567 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:06.577 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-11-16T20:24:06.618 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:06.620+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:06.673 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T20:24:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T20:24:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T20:24:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-11-16T20:24:06.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-11-16T20:24:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-11-16T20:24:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-11-16T20:24:06.676 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-11-16T20:24:06.691 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:06.690+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:06.695 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:06.695 INFO:teuthology.orchestra.run.smithi013.stdout: "size": 20, 2020-11-16T20:24:06.695 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 600, 2020-11-16T20:24:06.696 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [] 2020-11-16T20:24:06.696 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:06.706 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T20:24:06.793 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:06.792+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:06.857 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:06.858 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:06.858 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:06.858 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:06.868 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T20:24:06.893 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:06.890+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:06.994 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:06.990+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:07.041 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:07.042 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:07.042 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:07.042 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:07.042 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:07.051 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T20:24:07.094 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:07.090+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:07.182 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:07.182 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:07.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:07.183 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2020-11-16T20:24:07.183 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:07.193 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T20:24:07.195 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:07.196+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:07.296 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:07.294+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:07.312 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:07.312 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:07.312 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:07.312 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:07.322 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T20:24:07.397 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:07.396+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:07.435 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:07.436 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:07.436 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:07.436 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:07.436 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:07.446 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T20:24:07.534 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:07.536+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:07.547 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'unique_pool_0' created 2020-11-16T20:24:07.562 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2020-11-16T20:24:07.600 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:07.598+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:07.633 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:1 1.0 1:08c0073d:::INTEL_SSDPED1D480GA_PHMB7476008K480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2020-11-16T20:24:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.96172, 2020-11-16T20:24:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1045939339999999, 2020-11-16T20:24:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1 2020-11-16T20:24:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5326000000000004e-05 2020-11-16T20:24:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675933+0000", 2020-11-16T20:24:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2425999999999999e-05 2020-11-16T20:24:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675946+0000", 2020-11-16T20:24:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3563000000000001e-05 2020-11-16T20:24:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675969+0000", 2020-11-16T20:24:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4087000000000001e-05 2020-11-16T20:24:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675983+0000", 2020-11-16T20:24:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011954499999999999 2020-11-16T20:24:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.676103+0000", 2020-11-16T20:24:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.76e-05 2020-11-16T20:24:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2020-11-16T20:24:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.676130+0000", 2020-11-16T20:24:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1039924210000001 2020-11-16T20:24:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.780123+0000", 2020-11-16T20:24:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033896600000000001 2020-11-16T20:24:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.780462+0000", 2020-11-16T20:24:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1045939339999999 2020-11-16T20:24:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:2 1.0 1:08c0073d:::INTEL_SSDPED1D480GA_PHMB7476008K480DGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e14)", 2020-11-16T20:24:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:39.781601+0000", 2020-11-16T20:24:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.855986876999999, 2020-11-16T20:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040026519999999998, 2020-11-16T20:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 2 2020-11-16T20:24:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781601+0000", 2020-11-16T20:24:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781601+0000", 2020-11-16T20:24:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2702999999999997e-05 2020-11-16T20:24:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781684+0000", 2020-11-16T20:24:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3437e-05 2020-11-16T20:24:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781697+0000", 2020-11-16T20:24:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3926000000000001e-05 2020-11-16T20:24:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781721+0000", 2020-11-16T20:24:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3176999999999999e-05 2020-11-16T20:24:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781734+0000", 2020-11-16T20:24:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2101000000000005e-05 2020-11-16T20:24:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781826+0000", 2020-11-16T20:24:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027495200000000002 2020-11-16T20:24:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.782101+0000", 2020-11-16T20:24:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025125000000000001 2020-11-16T20:24:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2020-11-16T20:24:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.782353+0000", 2020-11-16T20:24:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015060620000000001 2020-11-16T20:24:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2020-11-16T20:24:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.783859+0000", 2020-11-16T20:24:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014876760000000001 2020-11-16T20:24:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2020-11-16T20:24:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785346+0000", 2020-11-16T20:24:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015005500000000001 2020-11-16T20:24:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785496+0000", 2020-11-16T20:24:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010731299999999999 2020-11-16T20:24:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785604+0000", 2020-11-16T20:24:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040026519999999998 2020-11-16T20:24:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4207.0:2 1.0 e14/13 ondisk, result = 0)", 2020-11-16T20:24:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:39.784936+0000", 2020-11-16T20:24:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.852652255999999, 2020-11-16T20:24:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067861400000000002, 2020-11-16T20:24:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.784936+0000", 2020-11-16T20:24:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.784936+0000", 2020-11-16T20:24:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011658399999999999 2020-11-16T20:24:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785052+0000", 2020-11-16T20:24:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.931e-05 2020-11-16T20:24:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785072+0000", 2020-11-16T20:24:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014155299999999999 2020-11-16T20:24:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785213+0000", 2020-11-16T20:24:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5877000000000001e-05 2020-11-16T20:24:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785229+0000", 2020-11-16T20:24:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5814000000000003e-05 2020-11-16T20:24:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785305+0000", 2020-11-16T20:24:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1569999999999998e-05 2020-11-16T20:24:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785326+0000", 2020-11-16T20:24:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028790599999999999 2020-11-16T20:24:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785614+0000", 2020-11-16T20:24:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067861400000000002 2020-11-16T20:24:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:3 1.0 1:97e83c24:::INTEL_SSDPEDMD400G4_PHFT620400QM400BGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e14)", 2020-11-16T20:24:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:39.913799+0000", 2020-11-16T20:24:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.723788937999998, 2020-11-16T20:24:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060138900000000003, 2020-11-16T20:24:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 3 2020-11-16T20:24:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.913799+0000", 2020-11-16T20:24:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.913799+0000", 2020-11-16T20:24:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108303 2020-11-16T20:24:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.913907+0000", 2020-11-16T20:24:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9094999999999999e-05 2020-11-16T20:24:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.913926+0000", 2020-11-16T20:24:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4683e-05 2020-11-16T20:24:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.913961+0000", 2020-11-16T20:24:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9031e-05 2020-11-16T20:24:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.913990+0000", 2020-11-16T20:24:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011956900000000001 2020-11-16T20:24:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.914110+0000", 2020-11-16T20:24:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029070799999999998 2020-11-16T20:24:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.914400+0000", 2020-11-16T20:24:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060138900000000003 2020-11-16T20:24:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:4 1.0 1:97e83c24:::INTEL_SSDPEDMD400G4_PHFT620400QM400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e14)", 2020-11-16T20:24:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.722531749000002, 2020-11-16T20:24:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045371300000000003, 2020-11-16T20:24:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 4 2020-11-16T20:24:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3885000000000002e-05 2020-11-16T20:24:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915130+0000", 2020-11-16T20:24:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.29e-05 2020-11-16T20:24:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915143+0000", 2020-11-16T20:24:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9257000000000001e-05 2020-11-16T20:24:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915172+0000", 2020-11-16T20:24:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9451999999999999e-05 2020-11-16T20:24:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915192+0000", 2020-11-16T20:24:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011178700000000001 2020-11-16T20:24:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915303+0000", 2020-11-16T20:24:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031313000000000001 2020-11-16T20:24:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915617+0000", 2020-11-16T20:24:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029708700000000002 2020-11-16T20:24:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2020-11-16T20:24:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915914+0000", 2020-11-16T20:24:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015209430000000001 2020-11-16T20:24:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2020-11-16T20:24:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.917435+0000", 2020-11-16T20:24:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018238799999999999 2020-11-16T20:24:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2020-11-16T20:24:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919258+0000", 2020-11-16T20:24:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208672 2020-11-16T20:24:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919467+0000", 2020-11-16T20:24:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012613699999999999 2020-11-16T20:24:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919593+0000", 2020-11-16T20:24:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045371300000000003 2020-11-16T20:24:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:07.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.685 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4207.0:4 1.0 e14/13 ondisk, result = 0)", 2020-11-16T20:24:07.686 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:39.918853+0000", 2020-11-16T20:24:07.686 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.718734455, 2020-11-16T20:24:07.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075138199999999998, 2020-11-16T20:24:07.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:07.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:07.686 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:07.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:07.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.918853+0000", 2020-11-16T20:24:07.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:07.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:07.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.918853+0000", 2020-11-16T20:24:07.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8823000000000001e-05 2020-11-16T20:24:07.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:07.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.918932+0000", 2020-11-16T20:24:07.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9409999999999999e-05 2020-11-16T20:24:07.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:07.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.918952+0000", 2020-11-16T20:24:07.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014011699999999999 2020-11-16T20:24:07.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:07.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919092+0000", 2020-11-16T20:24:07.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7479999999999999e-05 2020-11-16T20:24:07.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:07.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919109+0000", 2020-11-16T20:24:07.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4891999999999999e-05 2020-11-16T20:24:07.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:07.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919204+0000", 2020-11-16T20:24:07.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6276999999999999e-05 2020-11-16T20:24:07.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:07.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919230+0000", 2020-11-16T20:24:07.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000374383 2020-11-16T20:24:07.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:07.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:07.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:07.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919605+0000", 2020-11-16T20:24:07.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00075138199999999998 2020-11-16T20:24:07.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:07.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:07.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:07.694 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:07.695 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T20:24:07.738 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:07.740+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:07.801 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:07.800+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:07.857 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T20:24:07.902 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:07.898+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:08.002 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:08.004+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:08.006 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:08.008 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:08.018 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T20:24:08.104 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:08.102+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:08.181 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:08.182 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:08.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:08.182 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2020-11-16T20:24:08.182 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:08.193 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-11-16T20:24:08.251 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:08.251+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:08.306 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:08.303+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:08.327 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:08.328 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:08.338 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-11-16T20:24:08.423 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:08.423+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:08.492 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:08.493 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:08.493 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:08.493 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:08.493 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:08.503 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-11-16T20:24:08.547 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:08.547+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:08.557 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rados' on pool 'unique_pool_0' 2020-11-16T20:24:08.577 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T20:24:08.609 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:08.606+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:08.691 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:08.691 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:08.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:08.691 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2020-11-16T20:24:08.691 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:08.693 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T20:24:08.703 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-11-16T20:24:08.747 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:08.746+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:08.810 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:08.811+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:08.830 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:08.830 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:08.830 INFO:teuthology.orchestra.run.smithi013.stdout: "num_ops": 0 2020-11-16T20:24:08.830 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:08.840 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-11-16T20:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:08.876 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:08.904 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:08.927 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:08.926+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:08.930 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:08.962 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:08.992 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:09.006 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:09.006 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:09.013 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:09.010+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:09.016 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-11-16T20:24:09.017 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:09.048 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:09.050 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T20:24:09.050 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T20:24:09.051 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T20:24:09.051 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-11-16T20:24:09.051 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-11-16T20:24:09.051 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-11-16T20:24:09.051 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-11-16T20:24:09.051 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-11-16T20:24:09.113 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:09.115+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:09.181 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:09.181 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:09.193 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-11-16T20:24:09.239 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:09.239+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:09.308 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:09.308 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:09.315 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:09.315+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:09.318 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-11-16T20:24:09.415 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:09.411+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:09.478 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:09.478 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:09.488 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-11-16T20:24:09.518 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-11-16T20:24:09.518 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T20:24:09.519 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.164229 2020-11-16T20:24:09.519 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0.16422945938698186 2020-11-16T20:24:09.531 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:09.527+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:09.552 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2020-11-16T20:24:09.575 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 min_size 2 2020-11-16T20:24:09.617 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:09.615+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:09.645 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:09.645 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:09.655 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-11-16T20:24:09.717 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:09.719+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:09.775 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:09.775 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:09.785 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-11-16T20:24:09.835 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:09.831+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:09.903 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:09.904 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:09.914 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-11-16T20:24:09.918 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:09.919+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:10.020 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:10.015+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:10.078 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:10.078 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:10.088 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-11-16T20:24:10.135 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:10.131+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:10.222 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:10.222+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:10.259 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:10.259 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:10.269 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-11-16T20:24:10.323 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:10.319+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:10.386 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:10.386 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:10.396 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-11-16T20:24:10.438 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:10.435+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:10.525 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:10.519+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:14.663 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T20:24:14.663 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T20:24:14.665 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.5 primary-affinity to 0.164229 (8107622) 2020-11-16T20:24:14.666 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 3 min_size to 2 2020-11-16T20:24:14.666 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:14.666 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:14.667 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:14.660+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:14.667 INFO:teuthology.orchestra.run.smithi092:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2020-11-16T20:24:14.668 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T20:24:14.730 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight read -> 100 2020-11-16T20:24:14.731 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight delete -> 50 2020-11-16T20:24:14.731 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight snap_create -> 50 2020-11-16T20:24:14.731 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight snap_remove -> 50 2020-11-16T20:24:14.731 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight rollback -> 50 2020-11-16T20:24:14.731 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight setattr -> 25 2020-11-16T20:24:14.731 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight rmattr -> 25 2020-11-16T20:24:14.732 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight copy_from -> 50 2020-11-16T20:24:14.732 INFO:tasks.rados.rados.0.smithi092.stdout:adding op weight append -> 100 2020-11-16T20:24:14.732 INFO:tasks.rados.rados.0.smithi092.stdout:ceph version 16.0.0-7219-g911cc357 (911cc35794d871071a8bebfb0919482263b1d18b) pacific (dev) 2020-11-16T20:24:14.732 INFO:tasks.rados.rados.0.smithi092.stdout:Configuration: 2020-11-16T20:24:14.732 INFO:tasks.rados.rados.0.smithi092.stdout: Number of operations: 4000 2020-11-16T20:24:14.732 INFO:tasks.rados.rados.0.smithi092.stdout: Number of objects: 50 2020-11-16T20:24:14.732 INFO:tasks.rados.rados.0.smithi092.stdout: Max in flight operations: 16 2020-11-16T20:24:14.732 INFO:tasks.rados.rados.0.smithi092.stdout: Object size (in bytes): 4000000 2020-11-16T20:24:14.733 INFO:tasks.rados.rados.0.smithi092.stdout: Write stride min: 400000 2020-11-16T20:24:14.733 INFO:tasks.rados.rados.0.smithi092.stdout: Write stride max: 800000 2020-11-16T20:24:14.764 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:14.765+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:14.837 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:14.838 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:14.849 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T20:24:14.863 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:14.864 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:14.860+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:14.888 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:14.912 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:14.934 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:14.963 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:14.965 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:14.965+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:14.971 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:14.972 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:14.982 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T20:24:14.992 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:15.023 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:15.052 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T20:24:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T20:24:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T20:24:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-11-16T20:24:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-11-16T20:24:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-11-16T20:24:15.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-11-16T20:24:15.055 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-11-16T20:24:15.108 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:15.109+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:15.110 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:15.111 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:15.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:15.111 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2020-11-16T20:24:15.111 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:15.121 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T20:24:15.168 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:15.168+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:15.269 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:15.269+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:15.286 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:15.286 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:15.286 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:15.287 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:15.296 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T20:24:15.369 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:15.368+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:15.460 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:15.460 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:15.460 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:15.461 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:15.461 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:15.470 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:15.469+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:15.470 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T20:24:15.522 INFO:tasks.rados.rados.0.smithi092.stdout:1: write initial oid 1 2020-11-16T20:24:15.522 INFO:tasks.rados.rados.0.smithi092.stdout:1: seq_num 0 ranges {0=24576} 2020-11-16T20:24:15.523 INFO:tasks.rados.rados.0.smithi092.stdout:1: writing smithi09216568-1 from 0 to 24576 tid 1 2020-11-16T20:24:15.523 INFO:tasks.rados.rados.0.smithi092.stdout:2: write initial oid 2 2020-11-16T20:24:15.523 INFO:tasks.rados.rados.0.smithi092.stdout:2: seq_num 1 ranges {0=24576} 2020-11-16T20:24:15.523 INFO:tasks.rados.rados.0.smithi092.stdout:2: writing smithi09216568-2 from 0 to 24576 tid 1 2020-11-16T20:24:15.523 INFO:tasks.rados.rados.0.smithi092.stdout:3: write initial oid 3 2020-11-16T20:24:15.523 INFO:tasks.rados.rados.0.smithi092.stdout:3: seq_num 2 ranges {0=40960} 2020-11-16T20:24:15.524 INFO:tasks.rados.rados.0.smithi092.stdout:3: writing smithi09216568-3 from 0 to 40960 tid 1 2020-11-16T20:24:15.524 INFO:tasks.rados.rados.0.smithi092.stdout:4: write initial oid 4 2020-11-16T20:24:15.524 INFO:tasks.rados.rados.0.smithi092.stdout:4: seq_num 3 ranges {0=57344} 2020-11-16T20:24:15.524 INFO:tasks.rados.rados.0.smithi092.stdout:4: writing smithi09216568-4 from 0 to 57344 tid 1 2020-11-16T20:24:15.525 INFO:tasks.rados.rados.0.smithi092.stdout:5: write initial oid 5 2020-11-16T20:24:15.525 INFO:tasks.rados.rados.0.smithi092.stdout:5: seq_num 4 ranges {0=73728} 2020-11-16T20:24:15.525 INFO:tasks.rados.rados.0.smithi092.stdout:5: writing smithi09216568-5 from 0 to 73728 tid 1 2020-11-16T20:24:15.525 INFO:tasks.rados.rados.0.smithi092.stdout:6: write initial oid 6 2020-11-16T20:24:15.525 INFO:tasks.rados.rados.0.smithi092.stdout:6: seq_num 5 ranges {0=90112} 2020-11-16T20:24:15.525 INFO:tasks.rados.rados.0.smithi092.stdout:6: writing smithi09216568-6 from 0 to 90112 tid 1 2020-11-16T20:24:15.526 INFO:tasks.rados.rados.0.smithi092.stdout:7: write initial oid 7 2020-11-16T20:24:15.526 INFO:tasks.rados.rados.0.smithi092.stdout:7: seq_num 6 ranges {0=106496} 2020-11-16T20:24:15.526 INFO:tasks.rados.rados.0.smithi092.stdout:7: writing smithi09216568-7 from 0 to 106496 tid 1 2020-11-16T20:24:15.526 INFO:tasks.rados.rados.0.smithi092.stdout:8: write initial oid 8 2020-11-16T20:24:15.526 INFO:tasks.rados.rados.0.smithi092.stdout:8: seq_num 7 ranges {0=122880} 2020-11-16T20:24:15.527 INFO:tasks.rados.rados.0.smithi092.stdout:8: writing smithi09216568-8 from 0 to 122880 tid 1 2020-11-16T20:24:15.527 INFO:tasks.rados.rados.0.smithi092.stdout:9: write initial oid 9 2020-11-16T20:24:15.527 INFO:tasks.rados.rados.0.smithi092.stdout:9: seq_num 8 ranges {0=139264} 2020-11-16T20:24:15.528 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2020-11-16T20:24:15.528 INFO:tasks.rados.rados.0.smithi092.stdout:9: writing smithi09216568-9 from 0 to 139264 tid 1 2020-11-16T20:24:15.528 INFO:tasks.rados.rados.0.smithi092.stdout:10: write initial oid 10 2020-11-16T20:24:15.528 INFO:tasks.rados.rados.0.smithi092.stdout:10: seq_num 9 ranges {0=155648} 2020-11-16T20:24:15.529 INFO:tasks.rados.rados.0.smithi092.stdout:10: writing smithi09216568-10 from 0 to 155648 tid 1 2020-11-16T20:24:15.529 INFO:tasks.rados.rados.0.smithi092.stdout:11: write initial oid 11 2020-11-16T20:24:15.529 INFO:tasks.rados.rados.0.smithi092.stdout:11: seq_num 10 ranges {0=172032} 2020-11-16T20:24:15.535 INFO:tasks.rados.rados.0.smithi092.stdout:11: writing smithi09216568-11 from 0 to 172032 tid 1 2020-11-16T20:24:15.535 INFO:tasks.rados.rados.0.smithi092.stdout:12: write initial oid 12 2020-11-16T20:24:15.535 INFO:tasks.rados.rados.0.smithi092.stdout:12: seq_num 11 ranges {0=188416} 2020-11-16T20:24:15.537 INFO:tasks.rados.rados.0.smithi092.stdout:12: writing smithi09216568-12 from 0 to 188416 tid 1 2020-11-16T20:24:15.537 INFO:tasks.rados.rados.0.smithi092.stdout:13: write initial oid 13 2020-11-16T20:24:15.537 INFO:tasks.rados.rados.0.smithi092.stdout:13: seq_num 12 ranges {0=204800} 2020-11-16T20:24:15.540 INFO:tasks.rados.rados.0.smithi092.stdout:13: writing smithi09216568-13 from 0 to 204800 tid 1 2020-11-16T20:24:15.540 INFO:tasks.rados.rados.0.smithi092.stdout:14: write initial oid 14 2020-11-16T20:24:15.540 INFO:tasks.rados.rados.0.smithi092.stdout:14: seq_num 13 ranges {0=221184} 2020-11-16T20:24:15.544 INFO:tasks.rados.rados.0.smithi092.stdout:14: writing smithi09216568-14 from 0 to 221184 tid 1 2020-11-16T20:24:15.545 INFO:tasks.rados.rados.0.smithi092.stdout:4: finishing write tid 1 to smithi09216568-4 2020-11-16T20:24:15.545 INFO:tasks.rados.rados.0.smithi092.stdout:3: finishing write tid 1 to smithi09216568-3 2020-11-16T20:24:15.545 INFO:tasks.rados.rados.0.smithi092.stdout:8: finishing write tid 1 to smithi09216568-8 2020-11-16T20:24:15.545 INFO:tasks.rados.rados.0.smithi092.stdout:15: write initial oid 15 2020-11-16T20:24:15.546 INFO:tasks.rados.rados.0.smithi092.stdout:9: finishing write tid 1 to smithi09216568-9 2020-11-16T20:24:15.546 INFO:tasks.rados.rados.0.smithi092.stdout:15: seq_num 14 ranges {0=237568} 2020-11-16T20:24:15.547 INFO:tasks.rados.rados.0.smithi092.stdout:15: writing smithi09216568-15 from 0 to 237568 tid 1 2020-11-16T20:24:15.548 INFO:tasks.rados.rados.0.smithi092.stdout:2: finishing write tid 1 to smithi09216568-2 2020-11-16T20:24:15.548 INFO:tasks.rados.rados.0.smithi092.stdout:16: write initial oid 16 2020-11-16T20:24:15.548 INFO:tasks.rados.rados.0.smithi092.stdout:16: seq_num 15 ranges {0=253952} 2020-11-16T20:24:15.550 INFO:tasks.rados.rados.0.smithi092.stdout:16: writing smithi09216568-16 from 0 to 253952 tid 1 2020-11-16T20:24:15.551 INFO:tasks.rados.rados.0.smithi092.stdout:6: finishing write tid 1 to smithi09216568-6 2020-11-16T20:24:15.551 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2020-11-16T20:24:15.552 INFO:tasks.rados.rados.0.smithi092.stdout:3: finishing write tid 2 to smithi09216568-3 2020-11-16T20:24:15.553 INFO:tasks.rados.rados.0.smithi092.stdout:5: finishing write tid 1 to smithi09216568-5 2020-11-16T20:24:15.553 INFO:tasks.rados.rados.0.smithi092.stdout:7: finishing write tid 1 to smithi09216568-7 2020-11-16T20:24:15.554 INFO:tasks.rados.rados.0.smithi092.stdout:13: finishing write tid 1 to smithi09216568-13 2020-11-16T20:24:15.554 INFO:tasks.rados.rados.0.smithi092.stdout:8: finishing write tid 2 to smithi09216568-8 2020-11-16T20:24:15.555 INFO:tasks.rados.rados.0.smithi092.stdout:10: finishing write tid 1 to smithi09216568-10 2020-11-16T20:24:15.556 INFO:tasks.rados.rados.0.smithi092.stdout:9: finishing write tid 2 to smithi09216568-9 2020-11-16T20:24:15.556 INFO:tasks.rados.rados.0.smithi092.stdout:2: finishing write tid 2 to smithi09216568-2 2020-11-16T20:24:15.556 INFO:tasks.rados.rados.0.smithi092.stdout:6: finishing write tid 2 to smithi09216568-6 2020-11-16T20:24:15.558 INFO:tasks.rados.rados.0.smithi092.stdout:11: finishing write tid 1 to smithi09216568-11 2020-11-16T20:24:15.558 INFO:tasks.rados.rados.0.smithi092.stdout:2: finishing write tid 3 to smithi09216568-2 2020-11-16T20:24:15.558 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2020-11-16T20:24:15.558 INFO:tasks.rados.rados.0.smithi092.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2020-11-16T20:24:15.559 INFO:tasks.rados.rados.0.smithi092.stdout:2: done (15 left) 2020-11-16T20:24:15.559 INFO:tasks.rados.rados.0.smithi092.stdout:17: write initial oid 17 2020-11-16T20:24:15.559 INFO:tasks.rados.rados.0.smithi092.stdout:17: seq_num 16 ranges {0=270336} 2020-11-16T20:24:15.561 INFO:tasks.rados.rados.0.smithi092.stdout:17: writing smithi09216568-17 from 0 to 270336 tid 1 2020-11-16T20:24:15.562 INFO:tasks.rados.rados.0.smithi092.stdout:4: finishing write tid 2 to smithi09216568-4 2020-11-16T20:24:15.562 INFO:tasks.rados.rados.0.smithi092.stdout:9: finishing write tid 3 to smithi09216568-9 2020-11-16T20:24:15.562 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2020-11-16T20:24:15.563 INFO:tasks.rados.rados.0.smithi092.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2020-11-16T20:24:15.563 INFO:tasks.rados.rados.0.smithi092.stdout:6: finishing write tid 3 to smithi09216568-6 2020-11-16T20:24:15.563 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2020-11-16T20:24:15.563 INFO:tasks.rados.rados.0.smithi092.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2020-11-16T20:24:15.563 INFO:tasks.rados.rados.0.smithi092.stdout:14: finishing write tid 1 to smithi09216568-14 2020-11-16T20:24:15.563 INFO:tasks.rados.rados.0.smithi092.stdout:12: finishing write tid 1 to smithi09216568-12 2020-11-16T20:24:15.563 INFO:tasks.rados.rados.0.smithi092.stdout:3: finishing write tid 3 to smithi09216568-3 2020-11-16T20:24:15.564 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2020-11-16T20:24:15.564 INFO:tasks.rados.rados.0.smithi092.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2020-11-16T20:24:15.564 INFO:tasks.rados.rados.0.smithi092.stdout:15: finishing write tid 1 to smithi09216568-15 2020-11-16T20:24:15.564 INFO:tasks.rados.rados.0.smithi092.stdout:8: finishing write tid 3 to smithi09216568-8 2020-11-16T20:24:15.564 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2020-11-16T20:24:15.564 INFO:tasks.rados.rados.0.smithi092.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2020-11-16T20:24:15.564 INFO:tasks.rados.rados.0.smithi092.stdout:13: finishing write tid 2 to smithi09216568-13 2020-11-16T20:24:15.565 INFO:tasks.rados.rados.0.smithi092.stdout:10: finishing write tid 2 to smithi09216568-10 2020-11-16T20:24:15.565 INFO:tasks.rados.rados.0.smithi092.stdout:7: finishing write tid 2 to smithi09216568-7 2020-11-16T20:24:15.565 INFO:tasks.rados.rados.0.smithi092.stdout:3: done (15 left) 2020-11-16T20:24:15.565 INFO:tasks.rados.rados.0.smithi092.stdout:6: done (14 left) 2020-11-16T20:24:15.565 INFO:tasks.rados.rados.0.smithi092.stdout:8: done (13 left) 2020-11-16T20:24:15.565 INFO:tasks.rados.rados.0.smithi092.stdout:9: done (12 left) 2020-11-16T20:24:15.565 INFO:tasks.rados.rados.0.smithi092.stdout:18: write initial oid 18 2020-11-16T20:24:15.566 INFO:tasks.rados.rados.0.smithi092.stdout:16: finishing write tid 1 to smithi09216568-16 2020-11-16T20:24:15.566 INFO:tasks.rados.rados.0.smithi092.stdout:18: seq_num 17 ranges {0=286720} 2020-11-16T20:24:15.566 INFO:tasks.rados.rados.0.smithi092.stdout:18: writing smithi09216568-18 from 0 to 286720 tid 1 2020-11-16T20:24:15.566 INFO:tasks.rados.rados.0.smithi092.stdout:19: write initial oid 19 2020-11-16T20:24:15.567 INFO:tasks.rados.rados.0.smithi092.stdout:5: finishing write tid 2 to smithi09216568-5 2020-11-16T20:24:15.567 INFO:tasks.rados.rados.0.smithi092.stdout:4: finishing write tid 3 to smithi09216568-4 2020-11-16T20:24:15.567 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2020-11-16T20:24:15.567 INFO:tasks.rados.rados.0.smithi092.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2020-11-16T20:24:15.567 INFO:tasks.rados.rados.0.smithi092.stdout:11: finishing write tid 2 to smithi09216568-11 2020-11-16T20:24:15.567 INFO:tasks.rados.rados.0.smithi092.stdout:13: finishing write tid 3 to smithi09216568-13 2020-11-16T20:24:15.568 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 13 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2020-11-16T20:24:15.568 INFO:tasks.rados.rados.0.smithi092.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2020-11-16T20:24:15.568 INFO:tasks.rados.rados.0.smithi092.stdout:7: finishing write tid 3 to smithi09216568-7 2020-11-16T20:24:15.568 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 7 v 4 (ObjNum 6 snap 0 seq_num 6) dirty exists 2020-11-16T20:24:15.568 INFO:tasks.rados.rados.0.smithi092.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2020-11-16T20:24:15.568 INFO:tasks.rados.rados.0.smithi092.stdout:14: finishing write tid 2 to smithi09216568-14 2020-11-16T20:24:15.568 INFO:tasks.rados.rados.0.smithi092.stdout:19: seq_num 18 ranges {0=303104} 2020-11-16T20:24:15.569 INFO:tasks.rados.rados.0.smithi092.stdout:19: writing smithi09216568-19 from 0 to 303104 tid 1 2020-11-16T20:24:15.569 INFO:tasks.rados.rados.0.smithi092.stdout:4: done (13 left) 2020-11-16T20:24:15.569 INFO:tasks.rados.rados.0.smithi092.stdout:7: done (12 left) 2020-11-16T20:24:15.570 INFO:tasks.rados.rados.0.smithi092.stdout:13: done (11 left) 2020-11-16T20:24:15.570 INFO:tasks.rados.rados.0.smithi092.stdout:20: write initial oid 20 2020-11-16T20:24:15.570 INFO:tasks.rados.rados.0.smithi092.stdout:20: seq_num 19 ranges {0=319488} 2020-11-16T20:24:15.570 INFO:tasks.rados.rados.0.smithi092.stdout:20: writing smithi09216568-20 from 0 to 319488 tid 1 2020-11-16T20:24:15.570 INFO:tasks.rados.rados.0.smithi092.stdout:21: write initial oid 21 2020-11-16T20:24:15.570 INFO:tasks.rados.rados.0.smithi092.stdout:21: seq_num 20 ranges {0=344064} 2020-11-16T20:24:15.572 INFO:tasks.rados.rados.0.smithi092.stdout:21: writing smithi09216568-21 from 0 to 344064 tid 1 2020-11-16T20:24:15.572 INFO:tasks.rados.rados.0.smithi092.stdout:22: write initial oid 22 2020-11-16T20:24:15.572 INFO:tasks.rados.rados.0.smithi092.stdout:14: finishing write tid 3 to smithi09216568-14 2020-11-16T20:24:15.572 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 14 v 4 (ObjNum 13 snap 0 seq_num 13) dirty exists 2020-11-16T20:24:15.573 INFO:tasks.rados.rados.0.smithi092.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2020-11-16T20:24:15.573 INFO:tasks.rados.rados.0.smithi092.stdout:5: finishing write tid 3 to smithi09216568-5 2020-11-16T20:24:15.573 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 4 (ObjNum 4 snap 0 seq_num 4) dirty exists 2020-11-16T20:24:15.573 INFO:tasks.rados.rados.0.smithi092.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2020-11-16T20:24:15.573 INFO:tasks.rados.rados.0.smithi092.stdout:11: finishing write tid 3 to smithi09216568-11 2020-11-16T20:24:15.573 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 5 (ObjNum 10 snap 0 seq_num 10) dirty exists 2020-11-16T20:24:15.573 INFO:tasks.rados.rados.0.smithi092.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2020-11-16T20:24:15.574 INFO:tasks.rados.rados.0.smithi092.stdout:10: finishing write tid 3 to smithi09216568-10 2020-11-16T20:24:15.574 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 10 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2020-11-16T20:24:15.574 INFO:tasks.rados.rados.0.smithi092.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2020-11-16T20:24:15.574 INFO:tasks.rados.rados.0.smithi092.stdout:15: finishing write tid 2 to smithi09216568-15 2020-11-16T20:24:15.574 INFO:tasks.rados.rados.0.smithi092.stdout:22: seq_num 21 ranges {0=360448} 2020-11-16T20:24:15.575 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:15.573+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:15.575 INFO:tasks.rados.rados.0.smithi092.stdout:22: writing smithi09216568-22 from 0 to 360448 tid 1 2020-11-16T20:24:15.575 INFO:tasks.rados.rados.0.smithi092.stdout:15: finishing write tid 3 to smithi09216568-15 2020-11-16T20:24:15.576 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2020-11-16T20:24:15.576 INFO:tasks.rados.rados.0.smithi092.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2020-11-16T20:24:15.576 INFO:tasks.rados.rados.0.smithi092.stdout:5: done (13 left) 2020-11-16T20:24:15.576 INFO:tasks.rados.rados.0.smithi092.stdout:10: done (12 left) 2020-11-16T20:24:15.576 INFO:tasks.rados.rados.0.smithi092.stdout:11: done (11 left) 2020-11-16T20:24:15.576 INFO:tasks.rados.rados.0.smithi092.stdout:14: done (10 left) 2020-11-16T20:24:15.577 INFO:tasks.rados.rados.0.smithi092.stdout:15: done (9 left) 2020-11-16T20:24:15.577 INFO:tasks.rados.rados.0.smithi092.stdout:23: write initial oid 23 2020-11-16T20:24:15.577 INFO:tasks.rados.rados.0.smithi092.stdout:23: seq_num 22 ranges {0=376832} 2020-11-16T20:24:15.578 INFO:tasks.rados.rados.0.smithi092.stdout:23: writing smithi09216568-23 from 0 to 376832 tid 1 2020-11-16T20:24:15.578 INFO:tasks.rados.rados.0.smithi092.stdout:24: write initial oid 24 2020-11-16T20:24:15.578 INFO:tasks.rados.rados.0.smithi092.stdout:24: seq_num 23 ranges {0=393216} 2020-11-16T20:24:15.581 INFO:tasks.rados.rados.0.smithi092.stdout:24: writing smithi09216568-24 from 0 to 393216 tid 1 2020-11-16T20:24:15.581 INFO:tasks.rados.rados.0.smithi092.stdout:16: finishing write tid 2 to smithi09216568-16 2020-11-16T20:24:15.581 INFO:tasks.rados.rados.0.smithi092.stdout:16: finishing write tid 3 to smithi09216568-16 2020-11-16T20:24:15.581 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2020-11-16T20:24:15.581 INFO:tasks.rados.rados.0.smithi092.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2020-11-16T20:24:15.581 INFO:tasks.rados.rados.0.smithi092.stdout:17: finishing write tid 1 to smithi09216568-17 2020-11-16T20:24:15.582 INFO:tasks.rados.rados.0.smithi092.stdout:16: done (10 left) 2020-11-16T20:24:15.582 INFO:tasks.rados.rados.0.smithi092.stdout:25: write initial oid 25 2020-11-16T20:24:15.582 INFO:tasks.rados.rados.0.smithi092.stdout:25: seq_num 24 ranges {0=409600} 2020-11-16T20:24:15.584 INFO:tasks.rados.rados.0.smithi092.stdout:25: writing smithi09216568-25 from 0 to 409600 tid 1 2020-11-16T20:24:15.584 INFO:tasks.rados.rados.0.smithi092.stdout:19: finishing write tid 1 to smithi09216568-19 2020-11-16T20:24:15.584 INFO:tasks.rados.rados.0.smithi092.stdout:18: finishing write tid 1 to smithi09216568-18 2020-11-16T20:24:15.585 INFO:tasks.rados.rados.0.smithi092.stdout:26: write initial oid 26 2020-11-16T20:24:15.585 INFO:tasks.rados.rados.0.smithi092.stdout:26: seq_num 25 ranges {0=425984} 2020-11-16T20:24:15.586 INFO:tasks.rados.rados.0.smithi092.stdout:26: writing smithi09216568-26 from 0 to 425984 tid 1 2020-11-16T20:24:15.587 INFO:tasks.rados.rados.0.smithi092.stdout:27: write initial oid 27 2020-11-16T20:24:15.587 INFO:tasks.rados.rados.0.smithi092.stdout:27: seq_num 26 ranges {0=442368} 2020-11-16T20:24:15.591 INFO:tasks.rados.rados.0.smithi092.stdout:27: writing smithi09216568-27 from 0 to 442368 tid 1 2020-11-16T20:24:15.591 INFO:tasks.rados.rados.0.smithi092.stdout:28: write initial oid 28 2020-11-16T20:24:15.591 INFO:tasks.rados.rados.0.smithi092.stdout:28: seq_num 27 ranges {0=458752} 2020-11-16T20:24:15.594 INFO:tasks.rados.rados.0.smithi092.stdout:28: writing smithi09216568-28 from 0 to 458752 tid 1 2020-11-16T20:24:15.595 INFO:tasks.rados.rados.0.smithi092.stdout:29: write initial oid 29 2020-11-16T20:24:15.595 INFO:tasks.rados.rados.0.smithi092.stdout:29: seq_num 28 ranges {0=475136} 2020-11-16T20:24:15.597 INFO:tasks.rados.rados.0.smithi092.stdout:29: writing smithi09216568-29 from 0 to 475136 tid 1 2020-11-16T20:24:15.598 INFO:tasks.rados.rados.0.smithi092.stdout:30: write initial oid 30 2020-11-16T20:24:15.598 INFO:tasks.rados.rados.0.smithi092.stdout:30: seq_num 29 ranges {0=491520} 2020-11-16T20:24:15.601 INFO:tasks.rados.rados.0.smithi092.stdout:30: writing smithi09216568-30 from 0 to 491520 tid 1 2020-11-16T20:24:15.602 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2020-11-16T20:24:15.602 INFO:tasks.rados.rados.0.smithi092.stdout:20: finishing write tid 1 to smithi09216568-20 2020-11-16T20:24:15.603 INFO:tasks.rados.rados.0.smithi092.stdout:21: finishing write tid 1 to smithi09216568-21 2020-11-16T20:24:15.603 INFO:tasks.rados.rados.0.smithi092.stdout:17: finishing write tid 2 to smithi09216568-17 2020-11-16T20:24:15.603 INFO:tasks.rados.rados.0.smithi092.stdout:22: finishing write tid 1 to smithi09216568-22 2020-11-16T20:24:15.603 INFO:tasks.rados.rados.0.smithi092.stdout:18: finishing write tid 2 to smithi09216568-18 2020-11-16T20:24:15.603 INFO:tasks.rados.rados.0.smithi092.stdout:19: finishing write tid 2 to smithi09216568-19 2020-11-16T20:24:15.603 INFO:tasks.rados.rados.0.smithi092.stdout:23: finishing write tid 1 to smithi09216568-23 2020-11-16T20:24:15.603 INFO:tasks.rados.rados.0.smithi092.stdout:20: finishing write tid 2 to smithi09216568-20 2020-11-16T20:24:15.604 INFO:tasks.rados.rados.0.smithi092.stdout:18: finishing write tid 3 to smithi09216568-18 2020-11-16T20:24:15.604 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 18 v 2 (ObjNum 17 snap 0 seq_num 17) dirty exists 2020-11-16T20:24:15.604 INFO:tasks.rados.rados.0.smithi092.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2020-11-16T20:24:15.604 INFO:tasks.rados.rados.0.smithi092.stdout:22: finishing write tid 2 to smithi09216568-22 2020-11-16T20:24:15.604 INFO:tasks.rados.rados.0.smithi092.stdout:24: finishing write tid 1 to smithi09216568-24 2020-11-16T20:24:15.604 INFO:tasks.rados.rados.0.smithi092.stdout:17: finishing write tid 3 to smithi09216568-17 2020-11-16T20:24:15.604 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 17 v 6 (ObjNum 16 snap 0 seq_num 16) dirty exists 2020-11-16T20:24:15.604 INFO:tasks.rados.rados.0.smithi092.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2020-11-16T20:24:15.605 INFO:tasks.rados.rados.0.smithi092.stdout:21: finishing write tid 2 to smithi09216568-21 2020-11-16T20:24:15.605 INFO:tasks.rados.rados.0.smithi092.stdout:17: done (15 left) 2020-11-16T20:24:15.605 INFO:tasks.rados.rados.0.smithi092.stdout:18: done (14 left) 2020-11-16T20:24:15.605 INFO:tasks.rados.rados.0.smithi092.stdout:31: write initial oid 31 2020-11-16T20:24:15.605 INFO:tasks.rados.rados.0.smithi092.stdout:31: seq_num 30 ranges {0=507904} 2020-11-16T20:24:15.608 INFO:tasks.rados.rados.0.smithi092.stdout:31: writing smithi09216568-31 from 0 to 507904 tid 1 2020-11-16T20:24:15.608 INFO:tasks.rados.rados.0.smithi092.stdout:32: write initial oid 32 2020-11-16T20:24:15.608 INFO:tasks.rados.rados.0.smithi092.stdout:32: seq_num 31 ranges {0=524288} 2020-11-16T20:24:15.609 INFO:tasks.rados.rados.0.smithi092.stdout:32: writing smithi09216568-32 from 0 to 524288 tid 1 2020-11-16T20:24:15.609 INFO:tasks.rados.rados.0.smithi092.stdout:20: finishing write tid 3 to smithi09216568-20 2020-11-16T20:24:15.609 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 20 v 3 (ObjNum 19 snap 0 seq_num 19) dirty exists 2020-11-16T20:24:15.609 INFO:tasks.rados.rados.0.smithi092.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2020-11-16T20:24:15.609 INFO:tasks.rados.rados.0.smithi092.stdout:22: finishing write tid 3 to smithi09216568-22 2020-11-16T20:24:15.610 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 22 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2020-11-16T20:24:15.610 INFO:tasks.rados.rados.0.smithi092.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2020-11-16T20:24:15.610 INFO:tasks.rados.rados.0.smithi092.stdout:21: finishing write tid 3 to smithi09216568-21 2020-11-16T20:24:15.610 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 21 v 5 (ObjNum 20 snap 0 seq_num 20) dirty exists 2020-11-16T20:24:15.610 INFO:tasks.rados.rados.0.smithi092.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2020-11-16T20:24:15.610 INFO:tasks.rados.rados.0.smithi092.stdout:23: finishing write tid 2 to smithi09216568-23 2020-11-16T20:24:15.610 INFO:tasks.rados.rados.0.smithi092.stdout:24: finishing write tid 2 to smithi09216568-24 2020-11-16T20:24:15.611 INFO:tasks.rados.rados.0.smithi092.stdout:23: finishing write tid 3 to smithi09216568-23 2020-11-16T20:24:15.611 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 8 (ObjNum 22 snap 0 seq_num 22) dirty exists 2020-11-16T20:24:15.611 INFO:tasks.rados.rados.0.smithi092.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2020-11-16T20:24:15.611 INFO:tasks.rados.rados.0.smithi092.stdout:20: done (15 left) 2020-11-16T20:24:15.611 INFO:tasks.rados.rados.0.smithi092.stdout:21: done (14 left) 2020-11-16T20:24:15.611 INFO:tasks.rados.rados.0.smithi092.stdout:22: done (13 left) 2020-11-16T20:24:15.611 INFO:tasks.rados.rados.0.smithi092.stdout:23: done (12 left) 2020-11-16T20:24:15.612 INFO:tasks.rados.rados.0.smithi092.stdout:33: write initial oid 33 2020-11-16T20:24:15.612 INFO:tasks.rados.rados.0.smithi092.stdout:33: seq_num 32 ranges {0=540672} 2020-11-16T20:24:15.612 INFO:tasks.rados.rados.0.smithi092.stdout:33: writing smithi09216568-33 from 0 to 540672 tid 1 2020-11-16T20:24:15.613 INFO:tasks.rados.rados.0.smithi092.stdout:24: finishing write tid 3 to smithi09216568-24 2020-11-16T20:24:15.613 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2020-11-16T20:24:15.613 INFO:tasks.rados.rados.0.smithi092.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2020-11-16T20:24:15.614 INFO:tasks.rados.rados.0.smithi092.stdout:25: finishing write tid 1 to smithi09216568-25 2020-11-16T20:24:15.614 INFO:tasks.rados.rados.0.smithi092.stdout:19: finishing write tid 3 to smithi09216568-19 2020-11-16T20:24:15.614 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 4 (ObjNum 18 snap 0 seq_num 18) dirty exists 2020-11-16T20:24:15.614 INFO:tasks.rados.rados.0.smithi092.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2020-11-16T20:24:15.614 INFO:tasks.rados.rados.0.smithi092.stdout:19: done (12 left) 2020-11-16T20:24:15.614 INFO:tasks.rados.rados.0.smithi092.stdout:24: done (11 left) 2020-11-16T20:24:15.614 INFO:tasks.rados.rados.0.smithi092.stdout:34: write initial oid 34 2020-11-16T20:24:15.615 INFO:tasks.rados.rados.0.smithi092.stdout:34: seq_num 33 ranges {0=557056} 2020-11-16T20:24:15.617 INFO:tasks.rados.rados.0.smithi092.stdout:34: writing smithi09216568-34 from 0 to 557056 tid 1 2020-11-16T20:24:15.617 INFO:tasks.rados.rados.0.smithi092.stdout:27: finishing write tid 1 to smithi09216568-27 2020-11-16T20:24:15.618 INFO:tasks.rados.rados.0.smithi092.stdout:35: write initial oid 35 2020-11-16T20:24:15.618 INFO:tasks.rados.rados.0.smithi092.stdout:35: seq_num 34 ranges {0=573440} 2020-11-16T20:24:15.622 INFO:tasks.rados.rados.0.smithi092.stdout:35: writing smithi09216568-35 from 0 to 573440 tid 1 2020-11-16T20:24:15.623 INFO:tasks.rados.rados.0.smithi092.stdout:36: write initial oid 36 2020-11-16T20:24:15.623 INFO:tasks.rados.rados.0.smithi092.stdout:36: seq_num 35 ranges {0=589824} 2020-11-16T20:24:15.627 INFO:tasks.rados.rados.0.smithi092.stdout:36: writing smithi09216568-36 from 0 to 589824 tid 1 2020-11-16T20:24:15.627 INFO:tasks.rados.rados.0.smithi092.stdout:37: write initial oid 37 2020-11-16T20:24:15.627 INFO:tasks.rados.rados.0.smithi092.stdout:37: seq_num 36 ranges {0=606208} 2020-11-16T20:24:15.631 INFO:tasks.rados.rados.0.smithi092.stdout:37: writing smithi09216568-37 from 0 to 606208 tid 1 2020-11-16T20:24:15.632 INFO:tasks.rados.rados.0.smithi092.stdout:38: write initial oid 38 2020-11-16T20:24:15.632 INFO:tasks.rados.rados.0.smithi092.stdout:38: seq_num 37 ranges {0=622592} 2020-11-16T20:24:15.636 INFO:tasks.rados.rados.0.smithi092.stdout:38: writing smithi09216568-38 from 0 to 622592 tid 1 2020-11-16T20:24:15.637 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2020-11-16T20:24:15.637 INFO:tasks.rados.rados.0.smithi092.stdout:32: finishing write tid 1 to smithi09216568-32 2020-11-16T20:24:15.637 INFO:tasks.rados.rados.0.smithi092.stdout:34: finishing write tid 1 to smithi09216568-34 2020-11-16T20:24:15.638 INFO:tasks.rados.rados.0.smithi092.stdout:28: finishing write tid 1 to smithi09216568-28 2020-11-16T20:24:15.638 INFO:tasks.rados.rados.0.smithi092.stdout:34: finishing write tid 2 to smithi09216568-34 2020-11-16T20:24:15.638 INFO:tasks.rados.rados.0.smithi092.stdout:31: finishing write tid 1 to smithi09216568-31 2020-11-16T20:24:15.638 INFO:tasks.rados.rados.0.smithi092.stdout:34: finishing write tid 3 to smithi09216568-34 2020-11-16T20:24:15.638 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2020-11-16T20:24:15.638 INFO:tasks.rados.rados.0.smithi092.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2020-11-16T20:24:15.639 INFO:tasks.rados.rados.0.smithi092.stdout:27: finishing write tid 2 to smithi09216568-27 2020-11-16T20:24:15.639 INFO:tasks.rados.rados.0.smithi092.stdout:27: finishing write tid 3 to smithi09216568-27 2020-11-16T20:24:15.639 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 27 v 3 (ObjNum 26 snap 0 seq_num 26) dirty exists 2020-11-16T20:24:15.639 INFO:tasks.rados.rados.0.smithi092.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2020-11-16T20:24:15.639 INFO:tasks.rados.rados.0.smithi092.stdout:31: finishing write tid 2 to smithi09216568-31 2020-11-16T20:24:15.639 INFO:tasks.rados.rados.0.smithi092.stdout:25: finishing write tid 2 to smithi09216568-25 2020-11-16T20:24:15.639 INFO:tasks.rados.rados.0.smithi092.stdout:28: finishing write tid 2 to smithi09216568-28 2020-11-16T20:24:15.639 INFO:tasks.rados.rados.0.smithi092.stdout:31: finishing write tid 3 to smithi09216568-31 2020-11-16T20:24:15.640 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 31 v 4 (ObjNum 30 snap 0 seq_num 30) dirty exists 2020-11-16T20:24:15.640 INFO:tasks.rados.rados.0.smithi092.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2020-11-16T20:24:15.640 INFO:tasks.rados.rados.0.smithi092.stdout:25: finishing write tid 3 to smithi09216568-25 2020-11-16T20:24:15.640 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 25 v 7 (ObjNum 24 snap 0 seq_num 24) dirty exists 2020-11-16T20:24:15.640 INFO:tasks.rados.rados.0.smithi092.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2020-11-16T20:24:15.640 INFO:tasks.rados.rados.0.smithi092.stdout:28: finishing write tid 3 to smithi09216568-28 2020-11-16T20:24:15.640 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 28 v 8 (ObjNum 27 snap 0 seq_num 27) dirty exists 2020-11-16T20:24:15.641 INFO:tasks.rados.rados.0.smithi092.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2020-11-16T20:24:15.641 INFO:tasks.rados.rados.0.smithi092.stdout:25: done (15 left) 2020-11-16T20:24:15.641 INFO:tasks.rados.rados.0.smithi092.stdout:27: done (14 left) 2020-11-16T20:24:15.641 INFO:tasks.rados.rados.0.smithi092.stdout:28: done (13 left) 2020-11-16T20:24:15.641 INFO:tasks.rados.rados.0.smithi092.stdout:31: done (12 left) 2020-11-16T20:24:15.641 INFO:tasks.rados.rados.0.smithi092.stdout:34: done (11 left) 2020-11-16T20:24:15.641 INFO:tasks.rados.rados.0.smithi092.stdout:39: write initial oid 39 2020-11-16T20:24:15.642 INFO:tasks.rados.rados.0.smithi092.stdout:39: seq_num 38 ranges {0=638976} 2020-11-16T20:24:15.644 INFO:tasks.rados.rados.0.smithi092.stdout:39: writing smithi09216568-39 from 0 to 638976 tid 1 2020-11-16T20:24:15.648 INFO:tasks.rados.rados.0.smithi092.stdout:32: finishing write tid 2 to smithi09216568-32 2020-11-16T20:24:15.648 INFO:tasks.rados.rados.0.smithi092.stdout:30: finishing write tid 1 to smithi09216568-30 2020-11-16T20:24:15.648 INFO:tasks.rados.rados.0.smithi092.stdout:33: finishing write tid 1 to smithi09216568-33 2020-11-16T20:24:15.648 INFO:tasks.rados.rados.0.smithi092.stdout:32: finishing write tid 3 to smithi09216568-32 2020-11-16T20:24:15.649 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 32 v 2 (ObjNum 31 snap 0 seq_num 31) dirty exists 2020-11-16T20:24:15.649 INFO:tasks.rados.rados.0.smithi092.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2020-11-16T20:24:15.649 INFO:tasks.rados.rados.0.smithi092.stdout:1: finishing write tid 1 to smithi09216568-1 2020-11-16T20:24:15.649 INFO:tasks.rados.rados.0.smithi092.stdout:36: finishing write tid 1 to smithi09216568-36 2020-11-16T20:24:15.649 INFO:tasks.rados.rados.0.smithi092.stdout:32: done (11 left) 2020-11-16T20:24:15.649 INFO:tasks.rados.rados.0.smithi092.stdout:40: write initial oid 40 2020-11-16T20:24:15.649 INFO:tasks.rados.rados.0.smithi092.stdout:40: seq_num 39 ranges {0=663552} 2020-11-16T20:24:15.650 INFO:tasks.rados.rados.0.smithi092.stdout:40: writing smithi09216568-40 from 0 to 663552 tid 1 2020-11-16T20:24:15.650 INFO:tasks.rados.rados.0.smithi092.stdout:41: write initial oid 41 2020-11-16T20:24:15.650 INFO:tasks.rados.rados.0.smithi092.stdout:41: seq_num 40 ranges {0=679936} 2020-11-16T20:24:15.654 INFO:tasks.rados.rados.0.smithi092.stdout:41: writing smithi09216568-41 from 0 to 679936 tid 1 2020-11-16T20:24:15.654 INFO:tasks.rados.rados.0.smithi092.stdout:42: write initial oid 42 2020-11-16T20:24:15.654 INFO:tasks.rados.rados.0.smithi092.stdout:42: seq_num 41 ranges {0=696320} 2020-11-16T20:24:15.660 INFO:tasks.rados.rados.0.smithi092.stdout:42: writing smithi09216568-42 from 0 to 696320 tid 1 2020-11-16T20:24:15.660 INFO:tasks.rados.rados.0.smithi092.stdout:43: write initial oid 43 2020-11-16T20:24:15.660 INFO:tasks.rados.rados.0.smithi092.stdout:43: seq_num 42 ranges {0=712704} 2020-11-16T20:24:15.665 INFO:tasks.rados.rados.0.smithi092.stdout:43: writing smithi09216568-43 from 0 to 712704 tid 1 2020-11-16T20:24:15.666 INFO:tasks.rados.rados.0.smithi092.stdout:44: write initial oid 44 2020-11-16T20:24:15.666 INFO:tasks.rados.rados.0.smithi092.stdout:44: seq_num 43 ranges {0=729088} 2020-11-16T20:24:15.671 INFO:tasks.rados.rados.0.smithi092.stdout:44: writing smithi09216568-44 from 0 to 729088 tid 1 2020-11-16T20:24:15.672 INFO:tasks.rados.rados.0.smithi092.stdout:36: finishing write tid 2 to smithi09216568-36 2020-11-16T20:24:15.673 INFO:tasks.rados.rados.0.smithi092.stdout:12: finishing write tid 2 to smithi09216568-12 2020-11-16T20:24:15.673 INFO:tasks.rados.rados.0.smithi092.stdout:30: finishing write tid 2 to smithi09216568-30 2020-11-16T20:24:15.673 INFO:tasks.rados.rados.0.smithi092.stdout:33: finishing write tid 2 to smithi09216568-33 2020-11-16T20:24:15.673 INFO:tasks.rados.rados.0.smithi092.stdout:30: finishing write tid 3 to smithi09216568-30 2020-11-16T20:24:15.673 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2020-11-16T20:24:15.673 INFO:tasks.rados.rados.0.smithi092.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2020-11-16T20:24:15.674 INFO:tasks.rados.rados.0.smithi092.stdout:33: finishing write tid 3 to smithi09216568-33 2020-11-16T20:24:15.674 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 33 v 3 (ObjNum 32 snap 0 seq_num 32) dirty exists 2020-11-16T20:24:15.674 INFO:tasks.rados.rados.0.smithi092.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2020-11-16T20:24:15.674 INFO:tasks.rados.rados.0.smithi092.stdout:29: finishing write tid 1 to smithi09216568-29 2020-11-16T20:24:15.674 INFO:tasks.rados.rados.0.smithi092.stdout:36: finishing write tid 3 to smithi09216568-36 2020-11-16T20:24:15.674 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 36 v 2 (ObjNum 35 snap 0 seq_num 35) dirty exists 2020-11-16T20:24:15.674 INFO:tasks.rados.rados.0.smithi092.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2020-11-16T20:24:15.675 INFO:tasks.rados.rados.0.smithi092.stdout:12: finishing write tid 3 to smithi09216568-12 2020-11-16T20:24:15.675 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2020-11-16T20:24:15.675 INFO:tasks.rados.rados.0.smithi092.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2020-11-16T20:24:15.675 INFO:tasks.rados.rados.0.smithi092.stdout:37: finishing write tid 1 to smithi09216568-37 2020-11-16T20:24:15.675 INFO:tasks.rados.rados.0.smithi092.stdout:35: finishing write tid 1 to smithi09216568-35 2020-11-16T20:24:15.675 INFO:tasks.rados.rados.0.smithi092.stdout:1: finishing write tid 2 to smithi09216568-1 2020-11-16T20:24:15.675 INFO:tasks.rados.rados.0.smithi092.stdout:38: finishing write tid 1 to smithi09216568-38 2020-11-16T20:24:15.675 INFO:tasks.rados.rados.0.smithi092.stdout:1: finishing write tid 3 to smithi09216568-1 2020-11-16T20:24:15.676 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2020-11-16T20:24:15.676 INFO:tasks.rados.rados.0.smithi092.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2020-11-16T20:24:15.676 INFO:tasks.rados.rados.0.smithi092.stdout:26: finishing write tid 1 to smithi09216568-26 2020-11-16T20:24:15.676 INFO:tasks.rados.rados.0.smithi092.stdout:1: done (15 left) 2020-11-16T20:24:15.676 INFO:tasks.rados.rados.0.smithi092.stdout:12: done (14 left) 2020-11-16T20:24:15.676 INFO:tasks.rados.rados.0.smithi092.stdout:30: done (13 left) 2020-11-16T20:24:15.676 INFO:tasks.rados.rados.0.smithi092.stdout:33: done (12 left) 2020-11-16T20:24:15.677 INFO:tasks.rados.rados.0.smithi092.stdout:36: done (11 left) 2020-11-16T20:24:15.677 INFO:tasks.rados.rados.0.smithi092.stdout:45: write initial oid 45 2020-11-16T20:24:15.677 INFO:tasks.rados.rados.0.smithi092.stdout:45: seq_num 44 ranges {0=745472} 2020-11-16T20:24:15.677 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:15.673+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:15.678 INFO:tasks.rados.rados.0.smithi092.stdout:45: writing smithi09216568-45 from 0 to 745472 tid 1 2020-11-16T20:24:15.678 INFO:tasks.rados.rados.0.smithi092.stdout:29: finishing write tid 2 to smithi09216568-29 2020-11-16T20:24:15.678 INFO:tasks.rados.rados.0.smithi092.stdout:29: finishing write tid 3 to smithi09216568-29 2020-11-16T20:24:15.678 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 29 v 9 (ObjNum 28 snap 0 seq_num 28) dirty exists 2020-11-16T20:24:15.679 INFO:tasks.rados.rados.0.smithi092.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2020-11-16T20:24:15.679 INFO:tasks.rados.rados.0.smithi092.stdout:40: finishing write tid 1 to smithi09216568-40 2020-11-16T20:24:15.679 INFO:tasks.rados.rados.0.smithi092.stdout:29: done (11 left) 2020-11-16T20:24:15.679 INFO:tasks.rados.rados.0.smithi092.stdout:46: write initial oid 46 2020-11-16T20:24:15.679 INFO:tasks.rados.rados.0.smithi092.stdout:46: seq_num 45 ranges {0=761856} 2020-11-16T20:24:15.683 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:15.683 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:15.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:15.684 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:15.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.684 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:1 1.0 1:08c0073d:::INTEL_SSDPED1D480GA_PHMB7476008K480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2020-11-16T20:24:15.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:15.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.007608464, 2020-11-16T20:24:15.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1045939339999999, 2020-11-16T20:24:15.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:15.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:15.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:15.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1 2020-11-16T20:24:15.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.686 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:15.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5326000000000004e-05 2020-11-16T20:24:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675933+0000", 2020-11-16T20:24:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2425999999999999e-05 2020-11-16T20:24:15.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675946+0000", 2020-11-16T20:24:15.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3563000000000001e-05 2020-11-16T20:24:15.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675969+0000", 2020-11-16T20:24:15.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4087000000000001e-05 2020-11-16T20:24:15.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675983+0000", 2020-11-16T20:24:15.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011954499999999999 2020-11-16T20:24:15.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.676103+0000", 2020-11-16T20:24:15.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.76e-05 2020-11-16T20:24:15.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2020-11-16T20:24:15.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.676130+0000", 2020-11-16T20:24:15.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1039924210000001 2020-11-16T20:24:15.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.780123+0000", 2020-11-16T20:24:15.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033896600000000001 2020-11-16T20:24:15.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.780462+0000", 2020-11-16T20:24:15.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1045939339999999 2020-11-16T20:24:15.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.693 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:2 1.0 1:08c0073d:::INTEL_SSDPED1D480GA_PHMB7476008K480DGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e14)", 2020-11-16T20:24:15.693 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:39.781601+0000", 2020-11-16T20:24:15.693 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.901875341, 2020-11-16T20:24:15.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040026519999999998, 2020-11-16T20:24:15.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:15.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:15.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:15.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:15.694 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 2 2020-11-16T20:24:15.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.695 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781601+0000", 2020-11-16T20:24:15.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781601+0000", 2020-11-16T20:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2702999999999997e-05 2020-11-16T20:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781684+0000", 2020-11-16T20:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3437e-05 2020-11-16T20:24:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781697+0000", 2020-11-16T20:24:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3926000000000001e-05 2020-11-16T20:24:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781721+0000", 2020-11-16T20:24:15.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3176999999999999e-05 2020-11-16T20:24:15.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781734+0000", 2020-11-16T20:24:15.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2101000000000005e-05 2020-11-16T20:24:15.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.781826+0000", 2020-11-16T20:24:15.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027495200000000002 2020-11-16T20:24:15.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:15.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.782101+0000", 2020-11-16T20:24:15.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025125000000000001 2020-11-16T20:24:15.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2020-11-16T20:24:15.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.782353+0000", 2020-11-16T20:24:15.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015060620000000001 2020-11-16T20:24:15.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2020-11-16T20:24:15.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.783859+0000", 2020-11-16T20:24:15.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014876760000000001 2020-11-16T20:24:15.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2020-11-16T20:24:15.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785346+0000", 2020-11-16T20:24:15.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015005500000000001 2020-11-16T20:24:15.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:15.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785496+0000", 2020-11-16T20:24:15.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010731299999999999 2020-11-16T20:24:15.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.785604+0000", 2020-11-16T20:24:15.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040026519999999998 2020-11-16T20:24:15.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.704 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:4 1.0 1:97e83c24:::INTEL_SSDPEDMD400G4_PHFT620400QM400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e14)", 2020-11-16T20:24:15.704 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:15.704 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.768420212999999, 2020-11-16T20:24:15.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045371300000000003, 2020-11-16T20:24:15.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.705 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:15.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:15.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:15.705 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:15.705 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 4 2020-11-16T20:24:15.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:15.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:15.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3885000000000002e-05 2020-11-16T20:24:15.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915130+0000", 2020-11-16T20:24:15.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.29e-05 2020-11-16T20:24:15.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915143+0000", 2020-11-16T20:24:15.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9257000000000001e-05 2020-11-16T20:24:15.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915172+0000", 2020-11-16T20:24:15.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9451999999999999e-05 2020-11-16T20:24:15.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915192+0000", 2020-11-16T20:24:15.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011178700000000001 2020-11-16T20:24:15.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915303+0000", 2020-11-16T20:24:15.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031313000000000001 2020-11-16T20:24:15.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:15.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915617+0000", 2020-11-16T20:24:15.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029708700000000002 2020-11-16T20:24:15.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2020-11-16T20:24:15.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915914+0000", 2020-11-16T20:24:15.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015209430000000001 2020-11-16T20:24:15.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2020-11-16T20:24:15.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.917435+0000", 2020-11-16T20:24:15.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018238799999999999 2020-11-16T20:24:15.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2020-11-16T20:24:15.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919258+0000", 2020-11-16T20:24:15.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208672 2020-11-16T20:24:15.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919467+0000", 2020-11-16T20:24:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012613699999999999 2020-11-16T20:24:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919593+0000", 2020-11-16T20:24:15.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045371300000000003 2020-11-16T20:24:15.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.715 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.715 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 31/23 ECSubWrite(tid=1, reqid=client.4444.0:22, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:15.715 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.534302+0000", 2020-11-16T20:24:15.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.14917430200000001, 2020-11-16T20:24:15.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004870625, 2020-11-16T20:24:15.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534302+0000", 2020-11-16T20:24:15.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534302+0000", 2020-11-16T20:24:15.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000195269 2020-11-16T20:24:15.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534497+0000", 2020-11-16T20:24:15.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5259e-05 2020-11-16T20:24:15.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534523+0000", 2020-11-16T20:24:15.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9455000000000002e-05 2020-11-16T20:24:15.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534622+0000", 2020-11-16T20:24:15.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0397000000000001e-05 2020-11-16T20:24:15.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534632+0000", 2020-11-16T20:24:15.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4876000000000006e-05 2020-11-16T20:24:15.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534707+0000", 2020-11-16T20:24:15.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3736e-05 2020-11-16T20:24:15.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534721+0000", 2020-11-16T20:24:15.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043959799999999999 2020-11-16T20:24:15.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539117+0000", 2020-11-16T20:24:15.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5652999999999997e-05 2020-11-16T20:24:15.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539173+0000", 2020-11-16T20:24:15.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004870625 2020-11-16T20:24:15.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.724 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 31/23 ECSubWrite(tid=3, reqid=client.4444.0:37, at_version=31'2, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:15.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.544872+0000", 2020-11-16T20:24:15.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.13860434199999999, 2020-11-16T20:24:15.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065869739999999998, 2020-11-16T20:24:15.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.724 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.544872+0000", 2020-11-16T20:24:15.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.544872+0000", 2020-11-16T20:24:15.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012931100000000001 2020-11-16T20:24:15.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545001+0000", 2020-11-16T20:24:15.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5837e-05 2020-11-16T20:24:15.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545027+0000", 2020-11-16T20:24:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1269999999999999e-05 2020-11-16T20:24:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545108+0000", 2020-11-16T20:24:15.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7259999999999997e-06 2020-11-16T20:24:15.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545118+0000", 2020-11-16T20:24:15.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9775000000000003e-05 2020-11-16T20:24:15.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545178+0000", 2020-11-16T20:24:15.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2670999999999999e-05 2020-11-16T20:24:15.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545191+0000", 2020-11-16T20:24:15.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062062649999999999 2020-11-16T20:24:15.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551397+0000", 2020-11-16T20:24:15.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2119000000000006e-05 2020-11-16T20:24:15.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551459+0000", 2020-11-16T20:24:15.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065869739999999998 2020-11-16T20:24:15.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.732 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 31/23 ECSubWrite(tid=5, reqid=client.4444.0:23, at_version=31'3, trim_to=0'0, roll_forward_to=31'1))", 2020-11-16T20:24:15.732 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.548970+0000", 2020-11-16T20:24:15.732 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.134506457, 2020-11-16T20:24:15.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003523338, 2020-11-16T20:24:15.732 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.732 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.733 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548970+0000", 2020-11-16T20:24:15.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548970+0000", 2020-11-16T20:24:15.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0234000000000006e-05 2020-11-16T20:24:15.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.549040+0000", 2020-11-16T20:24:15.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5442000000000001e-05 2020-11-16T20:24:15.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.549066+0000", 2020-11-16T20:24:15.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8528000000000003e-05 2020-11-16T20:24:15.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.549164+0000", 2020-11-16T20:24:15.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1031e-05 2020-11-16T20:24:15.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.549175+0000", 2020-11-16T20:24:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1795e-05 2020-11-16T20:24:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.549237+0000", 2020-11-16T20:24:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4911e-05 2020-11-16T20:24:15.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.549252+0000", 2020-11-16T20:24:15.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022236859999999999 2020-11-16T20:24:15.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551476+0000", 2020-11-16T20:24:15.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010177109999999999 2020-11-16T20:24:15.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.552493+0000", 2020-11-16T20:24:15.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003523338 2020-11-16T20:24:15.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=3, reqid=client.4444.0:43, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:15.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.553831+0000", 2020-11-16T20:24:15.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.12964551599999999, 2020-11-16T20:24:15.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052721310000000002, 2020-11-16T20:24:15.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.741 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.553831+0000", 2020-11-16T20:24:15.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.553831+0000", 2020-11-16T20:24:15.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087011200000000003 2020-11-16T20:24:15.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554701+0000", 2020-11-16T20:24:15.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2575e-05 2020-11-16T20:24:15.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554724+0000", 2020-11-16T20:24:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8412999999999996e-05 2020-11-16T20:24:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554802+0000", 2020-11-16T20:24:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2260000000000005e-06 2020-11-16T20:24:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554810+0000", 2020-11-16T20:24:15.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2891000000000004e-05 2020-11-16T20:24:15.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554883+0000", 2020-11-16T20:24:15.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1015999999999999e-05 2020-11-16T20:24:15.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554894+0000", 2020-11-16T20:24:15.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004148746 2020-11-16T20:24:15.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.559043+0000", 2020-11-16T20:24:15.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0152000000000003e-05 2020-11-16T20:24:15.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.559103+0000", 2020-11-16T20:24:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052721310000000002 2020-11-16T20:24:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 31/23 ECSubWrite(tid=7, reqid=client.4444.0:38, at_version=31'4, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:15.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.557239+0000", 2020-11-16T20:24:15.749 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.126237183, 2020-11-16T20:24:15.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043101739999999999, 2020-11-16T20:24:15.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557239+0000", 2020-11-16T20:24:15.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557239+0000", 2020-11-16T20:24:15.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3023999999999998e-05 2020-11-16T20:24:15.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557302+0000", 2020-11-16T20:24:15.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4273e-05 2020-11-16T20:24:15.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557327+0000", 2020-11-16T20:24:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2374999999999997e-05 2020-11-16T20:24:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557409+0000", 2020-11-16T20:24:15.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3429999999999996e-06 2020-11-16T20:24:15.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557417+0000", 2020-11-16T20:24:15.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031612989999999998 2020-11-16T20:24:15.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560579+0000", 2020-11-16T20:24:15.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9585999999999997e-05 2020-11-16T20:24:15.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560618+0000", 2020-11-16T20:24:15.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086567000000000002 2020-11-16T20:24:15.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.561484+0000", 2020-11-16T20:24:15.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5604000000000004e-05 2020-11-16T20:24:15.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.561549+0000", 2020-11-16T20:24:15.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043101739999999999 2020-11-16T20:24:15.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=15, reqid=client.4444.0:61, at_version=31'3, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.581005+0000", 2020-11-16T20:24:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.102471726, 2020-11-16T20:24:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059672299999999996, 2020-11-16T20:24:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.757 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581005+0000", 2020-11-16T20:24:15.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581005+0000", 2020-11-16T20:24:15.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026132899999999998 2020-11-16T20:24:15.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581266+0000", 2020-11-16T20:24:15.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4717000000000001e-05 2020-11-16T20:24:15.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581291+0000", 2020-11-16T20:24:15.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6506000000000002e-05 2020-11-16T20:24:15.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581367+0000", 2020-11-16T20:24:15.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2019999999999993e-06 2020-11-16T20:24:15.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581376+0000", 2020-11-16T20:24:15.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1564999999999998e-05 2020-11-16T20:24:15.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581438+0000", 2020-11-16T20:24:15.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4117999999999999e-05 2020-11-16T20:24:15.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581452+0000", 2020-11-16T20:24:15.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054574990000000002 2020-11-16T20:24:15.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.586910+0000", 2020-11-16T20:24:15.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2293999999999997e-05 2020-11-16T20:24:15.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.586972+0000", 2020-11-16T20:24:15.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059672299999999996 2020-11-16T20:24:15.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.764 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.765 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=18, reqid=client.4444.0:70, at_version=31'4, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:15.765 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.588330+0000", 2020-11-16T20:24:15.765 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.095146883000000002, 2020-11-16T20:24:15.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012654103, 2020-11-16T20:24:15.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.766 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588330+0000", 2020-11-16T20:24:15.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588330+0000", 2020-11-16T20:24:15.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001257293 2020-11-16T20:24:15.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589587+0000", 2020-11-16T20:24:15.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7559000000000002e-05 2020-11-16T20:24:15.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589614+0000", 2020-11-16T20:24:15.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6135000000000002e-05 2020-11-16T20:24:15.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589690+0000", 2020-11-16T20:24:15.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0029e-05 2020-11-16T20:24:15.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589701+0000", 2020-11-16T20:24:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2075000000000003e-05 2020-11-16T20:24:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589783+0000", 2020-11-16T20:24:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8433000000000001e-05 2020-11-16T20:24:15.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589801+0000", 2020-11-16T20:24:15.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011126036000000001 2020-11-16T20:24:15.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600927+0000", 2020-11-16T20:24:15.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6542999999999997e-05 2020-11-16T20:24:15.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600984+0000", 2020-11-16T20:24:15.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012654103 2020-11-16T20:24:15.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=20, reqid=client.4444.0:62, at_version=31'5, trim_to=0'0, roll_forward_to=31'3))", 2020-11-16T20:24:15.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.598997+0000", 2020-11-16T20:24:15.773 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.084479849999999995, 2020-11-16T20:24:15.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027472500000000001, 2020-11-16T20:24:15.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.598997+0000", 2020-11-16T20:24:15.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.598997+0000", 2020-11-16T20:24:15.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1893000000000001e-05 2020-11-16T20:24:15.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.599028+0000", 2020-11-16T20:24:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9672000000000001e-05 2020-11-16T20:24:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.599048+0000", 2020-11-16T20:24:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5043999999999997e-05 2020-11-16T20:24:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.599103+0000", 2020-11-16T20:24:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3499999999999999e-06 2020-11-16T20:24:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.599110+0000", 2020-11-16T20:24:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6313e-05 2020-11-16T20:24:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.599177+0000", 2020-11-16T20:24:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5175e-05 2020-11-16T20:24:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.599192+0000", 2020-11-16T20:24:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024894209999999999 2020-11-16T20:24:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601681+0000", 2020-11-16T20:24:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2382e-05 2020-11-16T20:24:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601744+0000", 2020-11-16T20:24:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027472500000000001 2020-11-16T20:24:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s2 31/23 ECSubWrite(tid=21, reqid=client.4444.0:79, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.600262+0000", 2020-11-16T20:24:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.083214440000000001, 2020-11-16T20:24:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013056373, 2020-11-16T20:24:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600262+0000", 2020-11-16T20:24:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600262+0000", 2020-11-16T20:24:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046778800000000001 2020-11-16T20:24:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600730+0000", 2020-11-16T20:24:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4491e-05 2020-11-16T20:24:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600754+0000", 2020-11-16T20:24:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2707999999999995e-05 2020-11-16T20:24:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600837+0000", 2020-11-16T20:24:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1890000000000001e-06 2020-11-16T20:24:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600844+0000", 2020-11-16T20:24:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7360999999999994e-05 2020-11-16T20:24:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600911+0000", 2020-11-16T20:24:15.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.113e-05 2020-11-16T20:24:15.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600923+0000", 2020-11-16T20:24:15.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012339085 2020-11-16T20:24:15.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613262+0000", 2020-11-16T20:24:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6620999999999999e-05 2020-11-16T20:24:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613318+0000", 2020-11-16T20:24:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013056373 2020-11-16T20:24:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.789 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=24, reqid=client.4444.0:71, at_version=31'6, trim_to=0'0, roll_forward_to=31'4))", 2020-11-16T20:24:15.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.602830+0000", 2020-11-16T20:24:15.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.08064665, 2020-11-16T20:24:15.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046210660000000001, 2020-11-16T20:24:15.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602830+0000", 2020-11-16T20:24:15.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602830+0000", 2020-11-16T20:24:15.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5114999999999997e-05 2020-11-16T20:24:15.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602885+0000", 2020-11-16T20:24:15.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2595999999999998e-05 2020-11-16T20:24:15.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602907+0000", 2020-11-16T20:24:15.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4755999999999999e-05 2020-11-16T20:24:15.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603002+0000", 2020-11-16T20:24:15.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0198e-05 2020-11-16T20:24:15.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603012+0000", 2020-11-16T20:24:15.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9153000000000001e-05 2020-11-16T20:24:15.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603072+0000", 2020-11-16T20:24:15.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3925e-05 2020-11-16T20:24:15.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603085+0000", 2020-11-16T20:24:15.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043104240000000002 2020-11-16T20:24:15.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.607396+0000", 2020-11-16T20:24:15.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4898999999999997e-05 2020-11-16T20:24:15.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.607451+0000", 2020-11-16T20:24:15.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046210660000000001 2020-11-16T20:24:15.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.fs1 31/23 ECSubWrite(tid=11, reqid=client.4444.0:94, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:15.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.614607+0000", 2020-11-16T20:24:15.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.068869748999999994, 2020-11-16T20:24:15.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007262976, 2020-11-16T20:24:15.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614607+0000", 2020-11-16T20:24:15.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614607+0000", 2020-11-16T20:24:15.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014703399999999999 2020-11-16T20:24:15.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614754+0000", 2020-11-16T20:24:15.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0153000000000001e-05 2020-11-16T20:24:15.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614784+0000", 2020-11-16T20:24:15.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9525999999999997e-05 2020-11-16T20:24:15.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614863+0000", 2020-11-16T20:24:15.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3206e-05 2020-11-16T20:24:15.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614877+0000", 2020-11-16T20:24:15.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4196000000000001e-05 2020-11-16T20:24:15.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614931+0000", 2020-11-16T20:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1345000000000001e-05 2020-11-16T20:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614942+0000", 2020-11-16T20:24:15.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068665319999999998 2020-11-16T20:24:15.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.621809+0000", 2020-11-16T20:24:15.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0983999999999998e-05 2020-11-16T20:24:15.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.621870+0000", 2020-11-16T20:24:15.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007262976 2020-11-16T20:24:15.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.805 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.806 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s2 31/23 ECSubWrite(tid=28, reqid=client.4444.0:91, at_version=31'2, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:15.806 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.614836+0000", 2020-11-16T20:24:15.806 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.068640166000000002, 2020-11-16T20:24:15.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017114507000000001, 2020-11-16T20:24:15.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614836+0000", 2020-11-16T20:24:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614836+0000", 2020-11-16T20:24:15.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043176289999999999 2020-11-16T20:24:15.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619154+0000", 2020-11-16T20:24:15.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5164000000000002e-05 2020-11-16T20:24:15.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619179+0000", 2020-11-16T20:24:15.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014133 2020-11-16T20:24:15.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619320+0000", 2020-11-16T20:24:15.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3549999999999994e-06 2020-11-16T20:24:15.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619330+0000", 2020-11-16T20:24:15.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5703000000000004e-05 2020-11-16T20:24:15.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619395+0000", 2020-11-16T20:24:15.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3317e-05 2020-11-16T20:24:15.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619409+0000", 2020-11-16T20:24:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012482895000000001 2020-11-16T20:24:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631892+0000", 2020-11-16T20:24:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9113999999999997e-05 2020-11-16T20:24:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631951+0000", 2020-11-16T20:24:15.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017114507000000001 2020-11-16T20:24:15.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.813 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.814 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s2 31/23 ECSubWrite(tid=30, reqid=client.4444.0:80, at_version=31'3, trim_to=0'0, roll_forward_to=31'1))", 2020-11-16T20:24:15.814 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.620547+0000", 2020-11-16T20:24:15.814 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.062929526999999999, 2020-11-16T20:24:15.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011500491, 2020-11-16T20:24:15.814 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620547+0000", 2020-11-16T20:24:15.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620547+0000", 2020-11-16T20:24:15.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3017999999999998e-05 2020-11-16T20:24:15.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620620+0000", 2020-11-16T20:24:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2254e-05 2020-11-16T20:24:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620652+0000", 2020-11-16T20:24:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4431000000000006e-05 2020-11-16T20:24:15.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620727+0000", 2020-11-16T20:24:15.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.554e-06 2020-11-16T20:24:15.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620734+0000", 2020-11-16T20:24:15.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039200349999999997 2020-11-16T20:24:15.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.624654+0000", 2020-11-16T20:24:15.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3179999999999999e-05 2020-11-16T20:24:15.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.624667+0000", 2020-11-16T20:24:15.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073281939999999997 2020-11-16T20:24:15.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631996+0000", 2020-11-16T20:24:15.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1824999999999999e-05 2020-11-16T20:24:15.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.632047+0000", 2020-11-16T20:24:15.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011500491 2020-11-16T20:24:15.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.822 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.fs1 31/23 ECSubWrite(tid=13, reqid=client.4444.0:94, at_version=0'0, trim_to=0'0, roll_forward_to=31'1))", 2020-11-16T20:24:15.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.628943+0000", 2020-11-16T20:24:15.823 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.054533379999999999, 2020-11-16T20:24:15.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031090850000000001, 2020-11-16T20:24:15.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.628943+0000", 2020-11-16T20:24:15.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.628943+0000", 2020-11-16T20:24:15.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2024999999999997e-05 2020-11-16T20:24:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.628995+0000", 2020-11-16T20:24:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6626999999999999e-05 2020-11-16T20:24:15.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629012+0000", 2020-11-16T20:24:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000164487 2020-11-16T20:24:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629176+0000", 2020-11-16T20:24:15.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1030000000000002e-06 2020-11-16T20:24:15.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629184+0000", 2020-11-16T20:24:15.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1509999999999999e-05 2020-11-16T20:24:15.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629246+0000", 2020-11-16T20:24:15.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3397999999999999e-05 2020-11-16T20:24:15.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629259+0000", 2020-11-16T20:24:15.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027379589999999999 2020-11-16T20:24:15.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631997+0000", 2020-11-16T20:24:15.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4975999999999997e-05 2020-11-16T20:24:15.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.632052+0000", 2020-11-16T20:24:15.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031090850000000001 2020-11-16T20:24:15.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.fs1 31/23 ECSubWrite(tid=14, reqid=client.4444.0:95, at_version=31'2, trim_to=0'0, roll_forward_to=31'1))", 2020-11-16T20:24:15.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.629045+0000", 2020-11-16T20:24:15.833 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.054430992999999997, 2020-11-16T20:24:15.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030425259999999998, 2020-11-16T20:24:15.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629045+0000", 2020-11-16T20:24:15.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629045+0000", 2020-11-16T20:24:15.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7113000000000002e-05 2020-11-16T20:24:15.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629083+0000", 2020-11-16T20:24:15.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8692999999999999e-05 2020-11-16T20:24:15.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629101+0000", 2020-11-16T20:24:15.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108014 2020-11-16T20:24:15.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629209+0000", 2020-11-16T20:24:15.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1669999999999998e-06 2020-11-16T20:24:15.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629214+0000", 2020-11-16T20:24:15.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000209006 2020-11-16T20:24:15.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629423+0000", 2020-11-16T20:24:15.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0729e-05 2020-11-16T20:24:15.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.629434+0000", 2020-11-16T20:24:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002619639 2020-11-16T20:24:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.632054+0000", 2020-11-16T20:24:15.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4165000000000002e-05 2020-11-16T20:24:15.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.632088+0000", 2020-11-16T20:24:15.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030425259999999998 2020-11-16T20:24:15.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.841 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.as1 31/23 ECSubWrite(tid=17, reqid=client.4444.0:106, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:15.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.635521+0000", 2020-11-16T20:24:15.842 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.047955838000000001, 2020-11-16T20:24:15.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089614870000000006, 2020-11-16T20:24:15.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635521+0000", 2020-11-16T20:24:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635521+0000", 2020-11-16T20:24:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000163449 2020-11-16T20:24:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635684+0000", 2020-11-16T20:24:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2008999999999998e-05 2020-11-16T20:24:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635716+0000", 2020-11-16T20:24:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111994 2020-11-16T20:24:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635828+0000", 2020-11-16T20:24:15.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6809999999999998e-06 2020-11-16T20:24:15.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635836+0000", 2020-11-16T20:24:15.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1889000000000003e-05 2020-11-16T20:24:15.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635898+0000", 2020-11-16T20:24:15.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2262000000000001e-05 2020-11-16T20:24:15.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:15.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635910+0000", 2020-11-16T20:24:15.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085164839999999995 2020-11-16T20:24:15.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:15.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.644426+0000", 2020-11-16T20:24:15.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5719e-05 2020-11-16T20:24:15.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.644482+0000", 2020-11-16T20:24:15.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089614870000000006 2020-11-16T20:24:15.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.fs1 31/23 ECSubRead(tid=16, to_read={3:f5f0c2ff:::smithi09216568-32:head=0,4096,0}, subchunks={3:f5f0c2ff:::smithi09216568-32:head=[0,1]}, attrs_to_read=))", 2020-11-16T20:24:15.850 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.642338+0000", 2020-11-16T20:24:15.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.041138198000000001, 2020-11-16T20:24:15.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026325350000000001, 2020-11-16T20:24:15.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:15.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:15.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:15.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:15.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642338+0000", 2020-11-16T20:24:15.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:15.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:15.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642338+0000", 2020-11-16T20:24:15.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1023999999999998e-05 2020-11-16T20:24:15.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:15.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642389+0000", 2020-11-16T20:24:15.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4098e-05 2020-11-16T20:24:15.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:15.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642403+0000", 2020-11-16T20:24:15.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100221 2020-11-16T20:24:15.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:15.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642504+0000", 2020-11-16T20:24:15.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4940000000000003e-06 2020-11-16T20:24:15.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:15.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642512+0000", 2020-11-16T20:24:15.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1617999999999999e-05 2020-11-16T20:24:15.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:15.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642564+0000", 2020-11-16T20:24:15.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024070799999999998 2020-11-16T20:24:15.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:15.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:15.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:15.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.644971+0000", 2020-11-16T20:24:15.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026325350000000001 2020-11-16T20:24:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:15.858 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:15.862 INFO:tasks.rados.rados.0.smithi092.stdout:46: writing smithi09216568-46 from 0 to 761856 tid 1 2020-11-16T20:24:15.862 INFO:tasks.rados.rados.0.smithi092.stdout:47: write initial oid 47 2020-11-16T20:24:15.863 INFO:tasks.rados.rados.0.smithi092.stdout:47: seq_num 46 ranges {0=778240} 2020-11-16T20:24:15.863 INFO:tasks.rados.rados.0.smithi092.stdout:47: writing smithi09216568-47 from 0 to 778240 tid 1 2020-11-16T20:24:15.863 INFO:tasks.rados.rados.0.smithi092.stdout:39: finishing write tid 1 to smithi09216568-39 2020-11-16T20:24:15.863 INFO:tasks.rados.rados.0.smithi092.stdout:38: finishing write tid 2 to smithi09216568-38 2020-11-16T20:24:15.863 INFO:tasks.rados.rados.0.smithi092.stdout:41: finishing write tid 1 to smithi09216568-41 2020-11-16T20:24:15.863 INFO:tasks.rados.rados.0.smithi092.stdout:39: finishing write tid 2 to smithi09216568-39 2020-11-16T20:24:15.863 INFO:tasks.rados.rados.0.smithi092.stdout:38: finishing write tid 3 to smithi09216568-38 2020-11-16T20:24:15.864 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 38 v 5 (ObjNum 37 snap 0 seq_num 37) dirty exists 2020-11-16T20:24:15.864 INFO:tasks.rados.rados.0.smithi092.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2020-11-16T20:24:15.864 INFO:tasks.rados.rados.0.smithi092.stdout:39: finishing write tid 3 to smithi09216568-39 2020-11-16T20:24:15.864 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 39 v 6 (ObjNum 38 snap 0 seq_num 38) dirty exists 2020-11-16T20:24:15.864 INFO:tasks.rados.rados.0.smithi092.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2020-11-16T20:24:15.864 INFO:tasks.rados.rados.0.smithi092.stdout:38: done (12 left) 2020-11-16T20:24:15.864 INFO:tasks.rados.rados.0.smithi092.stdout:39: done (11 left) 2020-11-16T20:24:15.865 INFO:tasks.rados.rados.0.smithi092.stdout:48: write initial oid 48 2020-11-16T20:24:15.865 INFO:tasks.rados.rados.0.smithi092.stdout:48: seq_num 47 ranges {0=794624} 2020-11-16T20:24:15.865 INFO:tasks.rados.rados.0.smithi092.stdout:48: writing smithi09216568-48 from 0 to 794624 tid 1 2020-11-16T20:24:15.865 INFO:tasks.rados.rados.0.smithi092.stdout:49: write initial oid 49 2020-11-16T20:24:15.865 INFO:tasks.rados.rados.0.smithi092.stdout:49: seq_num 48 ranges {0=409600,409600=401408} 2020-11-16T20:24:15.865 INFO:tasks.rados.rados.0.smithi092.stdout:49: writing smithi09216568-49 from 0 to 409600 tid 1 2020-11-16T20:24:15.865 INFO:tasks.rados.rados.0.smithi092.stdout:49: writing smithi09216568-49 from 409600 to 811008 tid 2 2020-11-16T20:24:15.866 INFO:tasks.rados.rados.0.smithi092.stdout:50: write initial oid 50 2020-11-16T20:24:15.866 INFO:tasks.rados.rados.0.smithi092.stdout:50: seq_num 49 ranges {0=425984,425984=401408} 2020-11-16T20:24:15.866 INFO:tasks.rados.rados.0.smithi092.stdout:50: writing smithi09216568-50 from 0 to 425984 tid 1 2020-11-16T20:24:15.866 INFO:tasks.rados.rados.0.smithi092.stdout:50: writing smithi09216568-50 from 425984 to 827392 tid 2 2020-11-16T20:24:15.866 INFO:tasks.rados.rados.0.smithi092.stdout:51: copy_from oid 11 from oid 6 current snap is 0 2020-11-16T20:24:15.866 INFO:tasks.rados.rados.0.smithi092.stdout:42: finishing write tid 1 to smithi09216568-42 2020-11-16T20:24:15.866 INFO:tasks.rados.rados.0.smithi092.stdout:52: rollback oid 2 current snap is 0 2020-11-16T20:24:15.867 INFO:tasks.rados.rados.0.smithi092.stdout:52: done (15 left) 2020-11-16T20:24:15.867 INFO:tasks.rados.rados.0.smithi092.stdout:53: read oid 20 snap -1 2020-11-16T20:24:15.867 INFO:tasks.rados.rados.0.smithi092.stdout:53: expect (ObjNum 19 snap 0 seq_num 19) 2020-11-16T20:24:15.867 INFO:tasks.rados.rados.0.smithi092.stdout: waiting on 16 2020-11-16T20:24:15.867 INFO:tasks.rados.rados.0.smithi092.stdout:43: finishing write tid 1 to smithi09216568-43 2020-11-16T20:24:15.867 INFO:tasks.rados.rados.0.smithi092.stdout:37: finishing write tid 2 to smithi09216568-37 2020-11-16T20:24:15.867 INFO:tasks.rados.rados.0.smithi092.stdout:40: finishing write tid 2 to smithi09216568-40 2020-11-16T20:24:15.868 INFO:tasks.rados.rados.0.smithi092.stdout:44: finishing write tid 1 to smithi09216568-44 2020-11-16T20:24:15.868 INFO:tasks.rados.rados.0.smithi092.stdout:40: finishing write tid 3 to smithi09216568-40 2020-11-16T20:24:15.868 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 40 v 11 (ObjNum 39 snap 0 seq_num 39) dirty exists 2020-11-16T20:24:15.868 INFO:tasks.rados.rados.0.smithi092.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2020-11-16T20:24:15.868 INFO:tasks.rados.rados.0.smithi092.stdout:43: finishing write tid 2 to smithi09216568-43 2020-11-16T20:24:15.868 INFO:tasks.rados.rados.0.smithi092.stdout:41: finishing write tid 2 to smithi09216568-41 2020-11-16T20:24:15.868 INFO:tasks.rados.rados.0.smithi092.stdout:37: finishing write tid 3 to smithi09216568-37 2020-11-16T20:24:15.869 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 37 v 4 (ObjNum 36 snap 0 seq_num 36) dirty exists 2020-11-16T20:24:15.869 INFO:tasks.rados.rados.0.smithi092.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2020-11-16T20:24:15.869 INFO:tasks.rados.rados.0.smithi092.stdout:26: finishing write tid 2 to smithi09216568-26 2020-11-16T20:24:15.869 INFO:tasks.rados.rados.0.smithi092.stdout:41: finishing write tid 3 to smithi09216568-41 2020-11-16T20:24:15.869 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 41 v 12 (ObjNum 40 snap 0 seq_num 40) dirty exists 2020-11-16T20:24:15.869 INFO:tasks.rados.rados.0.smithi092.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2020-11-16T20:24:15.870 INFO:tasks.rados.rados.0.smithi092.stdout:37: done (15 left) 2020-11-16T20:24:15.870 INFO:tasks.rados.rados.0.smithi092.stdout:40: done (14 left) 2020-11-16T20:24:15.870 INFO:tasks.rados.rados.0.smithi092.stdout:41: done (13 left) 2020-11-16T20:24:15.870 INFO:tasks.rados.rados.0.smithi092.stdout:54: snap_create 2020-11-16T20:24:15.870 INFO:tasks.rados.rados.0.smithi092.stdout:47: finishing write tid 1 to smithi09216568-47 2020-11-16T20:24:15.870 INFO:tasks.rados.rados.0.smithi092.stdout:45: finishing write tid 1 to smithi09216568-45 2020-11-16T20:24:15.871 INFO:tasks.rados.rados.0.smithi092.stdout:42: finishing write tid 2 to smithi09216568-42 2020-11-16T20:24:15.871 INFO:tasks.rados.rados.0.smithi092.stdout:53: expect (ObjNum 19 snap 0 seq_num 19) 2020-11-16T20:24:15.871 INFO:tasks.rados.rados.0.smithi092.stdout:44: finishing write tid 2 to smithi09216568-44 2020-11-16T20:24:15.871 INFO:tasks.rados.rados.0.smithi092.stdout:26: finishing write tid 3 to smithi09216568-26 2020-11-16T20:24:15.871 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 26 v 7 (ObjNum 25 snap 0 seq_num 25) dirty exists 2020-11-16T20:24:15.871 INFO:tasks.rados.rados.0.smithi092.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2020-11-16T20:24:15.871 INFO:tasks.rados.rados.0.smithi092.stdout:43: finishing write tid 3 to smithi09216568-43 2020-11-16T20:24:15.872 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 43 v 5 (ObjNum 42 snap 0 seq_num 42) dirty exists 2020-11-16T20:24:15.872 INFO:tasks.rados.rados.0.smithi092.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2020-11-16T20:24:15.872 INFO:tasks.rados.rados.0.smithi092.stdout:47: finishing write tid 2 to smithi09216568-47 2020-11-16T20:24:15.872 INFO:tasks.rados.rados.0.smithi092.stdout:45: finishing write tid 2 to smithi09216568-45 2020-11-16T20:24:15.872 INFO:tasks.rados.rados.0.smithi092.stdout:47: finishing write tid 3 to smithi09216568-47 2020-11-16T20:24:15.872 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 47 v 4 (ObjNum 46 snap 0 seq_num 46) dirty exists 2020-11-16T20:24:15.872 INFO:tasks.rados.rados.0.smithi092.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2020-11-16T20:24:15.872 INFO:tasks.rados.rados.0.smithi092.stdout:49: finishing write tid 1 to smithi09216568-49 2020-11-16T20:24:15.873 INFO:tasks.rados.rados.0.smithi092.stdout:42: finishing write tid 3 to smithi09216568-42 2020-11-16T20:24:15.873 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 42 v 9 (ObjNum 41 snap 0 seq_num 41) dirty exists 2020-11-16T20:24:15.873 INFO:tasks.rados.rados.0.smithi092.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2020-11-16T20:24:15.873 INFO:tasks.rados.rados.0.smithi092.stdout:48: finishing write tid 1 to smithi09216568-48 2020-11-16T20:24:15.873 INFO:tasks.rados.rados.0.smithi092.stdout:49: finishing write tid 2 to smithi09216568-49 2020-11-16T20:24:15.873 INFO:tasks.rados.rados.0.smithi092.stdout:46: finishing write tid 1 to smithi09216568-46 2020-11-16T20:24:15.873 INFO:tasks.rados.rados.0.smithi092.stdout:50: finishing write tid 1 to smithi09216568-50 2020-11-16T20:24:15.874 INFO:tasks.rados.rados.0.smithi092.stdout:49: finishing write tid 3 to smithi09216568-49 2020-11-16T20:24:15.874 INFO:tasks.rados.rados.0.smithi092.stdout:35: finishing write tid 2 to smithi09216568-35 2020-11-16T20:24:15.874 INFO:tasks.rados.rados.0.smithi092.stdout:49: finishing write tid 4 to smithi09216568-49 2020-11-16T20:24:15.874 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2020-11-16T20:24:15.874 INFO:tasks.rados.rados.0.smithi092.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2020-11-16T20:24:15.874 INFO:tasks.rados.rados.0.smithi092.stdout:46: finishing write tid 2 to smithi09216568-46 2020-11-16T20:24:15.874 INFO:tasks.rados.rados.0.smithi092.stdout:35: finishing write tid 3 to smithi09216568-35 2020-11-16T20:24:15.875 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 35 v 10 (ObjNum 34 snap 0 seq_num 34) dirty exists 2020-11-16T20:24:15.875 INFO:tasks.rados.rados.0.smithi092.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2020-11-16T20:24:15.875 INFO:tasks.rados.rados.0.smithi092.stdout:46: finishing write tid 3 to smithi09216568-46 2020-11-16T20:24:15.875 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2020-11-16T20:24:15.875 INFO:tasks.rados.rados.0.smithi092.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2020-11-16T20:24:15.875 INFO:tasks.rados.rados.0.smithi092.stdout:48: finishing write tid 2 to smithi09216568-48 2020-11-16T20:24:15.875 INFO:tasks.rados.rados.0.smithi092.stdout:50: finishing write tid 2 to smithi09216568-50 2020-11-16T20:24:15.875 INFO:tasks.rados.rados.0.smithi092.stdout:50: finishing write tid 3 to smithi09216568-50 2020-11-16T20:24:15.876 INFO:tasks.rados.rados.0.smithi092.stdout:48: finishing write tid 3 to smithi09216568-48 2020-11-16T20:24:15.876 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 48 v 9 (ObjNum 47 snap 0 seq_num 47) dirty exists 2020-11-16T20:24:15.876 INFO:tasks.rados.rados.0.smithi092.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2020-11-16T20:24:15.876 INFO:tasks.rados.rados.0.smithi092.stdout:44: finishing write tid 3 to smithi09216568-44 2020-11-16T20:24:15.876 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 44 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2020-11-16T20:24:15.876 INFO:tasks.rados.rados.0.smithi092.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2020-11-16T20:24:15.876 INFO:tasks.rados.rados.0.smithi092.stdout:51: finishing copy_from to smithi09216568-11 2020-11-16T20:24:15.877 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2020-11-16T20:24:15.877 INFO:tasks.rados.rados.0.smithi092.stdout:51: finishing copy_from racing read to smithi09216568-11 2020-11-16T20:24:15.877 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T20:24:15.878 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:15.860+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:15.961 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:15.960+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:16.030 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:16.030 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:16.031 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:16.041 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T20:24:16.062 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:16.060+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:16.100 INFO:tasks.rados.rados.0.smithi092.stdout:50: finishing write tid 4 to smithi09216568-50 2020-11-16T20:24:16.101 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 11 (ObjNum 49 snap 0 seq_num 49) dirty exists 2020-11-16T20:24:16.101 INFO:tasks.rados.rados.0.smithi092.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2020-11-16T20:24:16.156 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:16.157 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:16.164 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:16.160+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:16.166 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T20:24:16.264 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:16.265+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:16.285 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.286 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 30/26 ECSubWrite(tid=0, reqid=client.4444.0:1, at_version=30'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.75378600600000001, 2020-11-16T20:24:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110371539, 2020-11-16T20:24:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:16.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1965999999999994e-05 2020-11-16T20:24:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532961+0000", 2020-11-16T20:24:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4882000000000002e-05 2020-11-16T20:24:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532996+0000", 2020-11-16T20:24:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.107939761 2020-11-16T20:24:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640936+0000", 2020-11-16T20:24:16.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3749999999999997e-06 2020-11-16T20:24:16.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640943+0000", 2020-11-16T20:24:16.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000229192 2020-11-16T20:24:16.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641172+0000", 2020-11-16T20:24:16.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3837e-05 2020-11-16T20:24:16.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641186+0000", 2020-11-16T20:24:16.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002004053 2020-11-16T20:24:16.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643190+0000", 2020-11-16T20:24:16.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0473000000000002e-05 2020-11-16T20:24:16.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643251+0000", 2020-11-16T20:24:16.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110371539 2020-11-16T20:24:16.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.294 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.295 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:54 3.5s0 3:aad7141b:::smithi09216568-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:16.295 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:16.295 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.71259129300000001, 2020-11-16T20:24:16.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027636214999999999, 2020-11-16T20:24:16.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 54 2020-11-16T20:24:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:16.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:16.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7353e-05 2020-11-16T20:24:16.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574111+0000", 2020-11-16T20:24:16.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1509999999999998e-06 2020-11-16T20:24:16.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574118+0000", 2020-11-16T20:24:16.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2278e-05 2020-11-16T20:24:16.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574131+0000", 2020-11-16T20:24:16.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3789999999999997e-06 2020-11-16T20:24:16.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574136+0000", 2020-11-16T20:24:16.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054445750000000001 2020-11-16T20:24:16.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.579581+0000", 2020-11-16T20:24:16.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117794 2020-11-16T20:24:16.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.579698+0000", 2020-11-16T20:24:16.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056929220000000004 2020-11-16T20:24:16.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.585391+0000", 2020-11-16T20:24:16.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106148 2020-11-16T20:24:16.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.585498+0000", 2020-11-16T20:24:16.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097071099999999997 2020-11-16T20:24:16.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.595205+0000", 2020-11-16T20:24:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013155099999999999 2020-11-16T20:24:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.595336+0000", 2020-11-16T20:24:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063739540000000003 2020-11-16T20:24:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601710+0000", 2020-11-16T20:24:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027636214999999999 2020-11-16T20:24:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.306 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:59 3.es0 3:746783df:::smithi09216568-20:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:16.306 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:16.306 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.71212416199999995, 2020-11-16T20:24:16.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026665233, 2020-11-16T20:24:16.307 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.307 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:16.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.307 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59 2020-11-16T20:24:16.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.308 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:16.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:16.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5383999999999997e-05 2020-11-16T20:24:16.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574596+0000", 2020-11-16T20:24:16.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999998e-06 2020-11-16T20:24:16.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574604+0000", 2020-11-16T20:24:16.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3088000000000001e-05 2020-11-16T20:24:16.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574617+0000", 2020-11-16T20:24:16.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1129999999999997e-06 2020-11-16T20:24:16.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574623+0000", 2020-11-16T20:24:16.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007628709 2020-11-16T20:24:16.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582252+0000", 2020-11-16T20:24:16.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011149099999999999 2020-11-16T20:24:16.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582363+0000", 2020-11-16T20:24:16.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069613909999999999 2020-11-16T20:24:16.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589325+0000", 2020-11-16T20:24:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0105000000000001e-05 2020-11-16T20:24:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589385+0000", 2020-11-16T20:24:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052031099999999997 2020-11-16T20:24:16.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589905+0000", 2020-11-16T20:24:16.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072848720000000004 2020-11-16T20:24:16.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.597190+0000", 2020-11-16T20:24:16.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00390829 2020-11-16T20:24:16.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:16.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601098+0000", 2020-11-16T20:24:16.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108199 2020-11-16T20:24:16.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601206+0000", 2020-11-16T20:24:16.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026665233 2020-11-16T20:24:16.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.317 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:60 3.es0 3:746783df:::smithi09216568-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:16.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:16.318 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.71199222100000004, 2020-11-16T20:24:16.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029540154999999998, 2020-11-16T20:24:16.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:16.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 60 2020-11-16T20:24:16.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:16.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:16.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9110000000000003e-05 2020-11-16T20:24:16.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574712+0000", 2020-11-16T20:24:16.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8739999999999997e-06 2020-11-16T20:24:16.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574719+0000", 2020-11-16T20:24:16.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2554e-05 2020-11-16T20:24:16.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574732+0000", 2020-11-16T20:24:16.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-06 2020-11-16T20:24:16.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574736+0000", 2020-11-16T20:24:16.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077321949999999999 2020-11-16T20:24:16.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582469+0000", 2020-11-16T20:24:16.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010602 2020-11-16T20:24:16.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582575+0000", 2020-11-16T20:24:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078916549999999992 2020-11-16T20:24:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.590466+0000", 2020-11-16T20:24:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107569 2020-11-16T20:24:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.590574+0000", 2020-11-16T20:24:16.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010689094999999999 2020-11-16T20:24:16.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601263+0000", 2020-11-16T20:24:16.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7796999999999999e-05 2020-11-16T20:24:16.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601361+0000", 2020-11-16T20:24:16.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028524959999999999 2020-11-16T20:24:16.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604213+0000", 2020-11-16T20:24:16.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029540154999999998 2020-11-16T20:24:16.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.329 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:65 3.es0 3:7fa2fe22:::smithi09216568-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:16.329 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.578706+0000", 2020-11-16T20:24:16.329 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.70795876300000005, 2020-11-16T20:24:16.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023176904000000002, 2020-11-16T20:24:16.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:16.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.330 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 65 2020-11-16T20:24:16.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.330 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578706+0000", 2020-11-16T20:24:16.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578706+0000", 2020-11-16T20:24:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5914000000000006e-05 2020-11-16T20:24:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578772+0000", 2020-11-16T20:24:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9780000000000005e-06 2020-11-16T20:24:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578780+0000", 2020-11-16T20:24:16.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5695000000000001e-05 2020-11-16T20:24:16.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578796+0000", 2020-11-16T20:24:16.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9989999999999997e-06 2020-11-16T20:24:16.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578802+0000", 2020-11-16T20:24:16.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095163260000000003 2020-11-16T20:24:16.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588318+0000", 2020-11-16T20:24:16.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117637 2020-11-16T20:24:16.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588436+0000", 2020-11-16T20:24:16.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042836439999999996 2020-11-16T20:24:16.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.592720+0000", 2020-11-16T20:24:16.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8870999999999996e-05 2020-11-16T20:24:16.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.592819+0000", 2020-11-16T20:24:16.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000502451 2020-11-16T20:24:16.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.593321+0000", 2020-11-16T20:24:16.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061881109999999996 2020-11-16T20:24:16.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.599509+0000", 2020-11-16T20:24:16.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002289265 2020-11-16T20:24:16.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601798+0000", 2020-11-16T20:24:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5012999999999994e-05 2020-11-16T20:24:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601883+0000", 2020-11-16T20:24:16.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023176904000000002 2020-11-16T20:24:16.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.341 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:66 3.es0 3:7fa2fe22:::smithi09216568-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:16.341 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.578915+0000", 2020-11-16T20:24:16.341 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.70775007300000004, 2020-11-16T20:24:16.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025562700000000001, 2020-11-16T20:24:16.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:16.341 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.342 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66 2020-11-16T20:24:16.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.342 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578915+0000", 2020-11-16T20:24:16.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578915+0000", 2020-11-16T20:24:16.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4132000000000003e-05 2020-11-16T20:24:16.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578969+0000", 2020-11-16T20:24:16.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0862e-05 2020-11-16T20:24:16.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578980+0000", 2020-11-16T20:24:16.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8485e-05 2020-11-16T20:24:16.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578999+0000", 2020-11-16T20:24:16.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9820000000000008e-06 2020-11-16T20:24:16.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.579007+0000", 2020-11-16T20:24:16.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097006060000000005 2020-11-16T20:24:16.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588707+0000", 2020-11-16T20:24:16.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010008100000000001 2020-11-16T20:24:16.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588807+0000", 2020-11-16T20:24:16.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047694950000000003 2020-11-16T20:24:16.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.593577+0000", 2020-11-16T20:24:16.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104916 2020-11-16T20:24:16.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.593682+0000", 2020-11-16T20:24:16.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085152460000000006 2020-11-16T20:24:16.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602197+0000", 2020-11-16T20:24:16.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8281000000000003e-05 2020-11-16T20:24:16.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602295+0000", 2020-11-16T20:24:16.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021826139999999998 2020-11-16T20:24:16.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604478+0000", 2020-11-16T20:24:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025562700000000001 2020-11-16T20:24:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.351 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s2 31/23 ECSubWrite(tid=15, reqid=client.4444.0:85, at_version=31'7, trim_to=0'0, roll_forward_to=31'5))", 2020-11-16T20:24:16.352 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:16.352 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.68088892700000003, 2020-11-16T20:24:16.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036849253999999998, 2020-11-16T20:24:16.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.352 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:16.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:16.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014849700000000001 2020-11-16T20:24:16.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605925+0000", 2020-11-16T20:24:16.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2458000000000001e-05 2020-11-16T20:24:16.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605947+0000", 2020-11-16T20:24:16.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5471999999999999e-05 2020-11-16T20:24:16.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606013+0000", 2020-11-16T20:24:16.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8510000000000001e-06 2020-11-16T20:24:16.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606020+0000", 2020-11-16T20:24:16.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000222365 2020-11-16T20:24:16.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606242+0000", 2020-11-16T20:24:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8460000000000003e-06 2020-11-16T20:24:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606252+0000", 2020-11-16T20:24:16.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036144568000000002 2020-11-16T20:24:16.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642396+0000", 2020-11-16T20:24:16.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022919699999999999 2020-11-16T20:24:16.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642626+0000", 2020-11-16T20:24:16.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036849253999999998 2020-11-16T20:24:16.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.359 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.360 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.8s2 31/23 ECSubWrite(tid=29, reqid=client.4444.0:74, at_version=31'7, trim_to=0'0, roll_forward_to=31'5))", 2020-11-16T20:24:16.360 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.612585+0000", 2020-11-16T20:24:16.360 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.67408063100000004, 2020-11-16T20:24:16.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023074130000000002, 2020-11-16T20:24:16.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.361 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612585+0000", 2020-11-16T20:24:16.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612585+0000", 2020-11-16T20:24:16.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4243999999999998e-05 2020-11-16T20:24:16.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612639+0000", 2020-11-16T20:24:16.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4958999999999999e-05 2020-11-16T20:24:16.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612664+0000", 2020-11-16T20:24:16.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5123999999999994e-05 2020-11-16T20:24:16.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612739+0000", 2020-11-16T20:24:16.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.712e-06 2020-11-16T20:24:16.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612747+0000", 2020-11-16T20:24:16.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110031 2020-11-16T20:24:16.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612857+0000", 2020-11-16T20:24:16.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2428e-05 2020-11-16T20:24:16.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612869+0000", 2020-11-16T20:24:16.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022728992999999999 2020-11-16T20:24:16.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635598+0000", 2020-11-16T20:24:16.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0639e-05 2020-11-16T20:24:16.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635659+0000", 2020-11-16T20:24:16.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023074130000000002 2020-11-16T20:24:16.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.367 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.3s1 31/23 ECSubWrite(tid=10, reqid=client.4444.0:88, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:16.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.613775+0000", 2020-11-16T20:24:16.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.672890349, 2020-11-16T20:24:16.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02647045, 2020-11-16T20:24:16.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613775+0000", 2020-11-16T20:24:16.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613775+0000", 2020-11-16T20:24:16.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021481899999999999 2020-11-16T20:24:16.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613990+0000", 2020-11-16T20:24:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6764e-05 2020-11-16T20:24:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614016+0000", 2020-11-16T20:24:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020432299999999999 2020-11-16T20:24:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614221+0000", 2020-11-16T20:24:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7100000000000001e-06 2020-11-16T20:24:16.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614227+0000", 2020-11-16T20:24:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9741999999999998e-05 2020-11-16T20:24:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614287+0000", 2020-11-16T20:24:16.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1365e-05 2020-11-16T20:24:16.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614299+0000", 2020-11-16T20:24:16.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025885973 2020-11-16T20:24:16.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640185+0000", 2020-11-16T20:24:16.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0754000000000001e-05 2020-11-16T20:24:16.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640245+0000", 2020-11-16T20:24:16.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02647045 2020-11-16T20:24:16.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.376 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.376 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.3s1 31/23 ECSubWrite(tid=12, reqid=client.4444.0:97, at_version=31'2, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:16.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.618268+0000", 2020-11-16T20:24:16.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.66839674999999998, 2020-11-16T20:24:16.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022057766999999999, 2020-11-16T20:24:16.377 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.377 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.377 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.618268+0000", 2020-11-16T20:24:16.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.618268+0000", 2020-11-16T20:24:16.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000198023 2020-11-16T20:24:16.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.618467+0000", 2020-11-16T20:24:16.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5882999999999999e-05 2020-11-16T20:24:16.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.618492+0000", 2020-11-16T20:24:16.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8656e-05 2020-11-16T20:24:16.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.618581+0000", 2020-11-16T20:24:16.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2790000000000001e-06 2020-11-16T20:24:16.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.618588+0000", 2020-11-16T20:24:16.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024670730000000002 2020-11-16T20:24:16.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.621055+0000", 2020-11-16T20:24:16.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1990999999999999e-05 2020-11-16T20:24:16.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.621067+0000", 2020-11-16T20:24:16.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019218731999999999 2020-11-16T20:24:16.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640286+0000", 2020-11-16T20:24:16.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0129999999999997e-05 2020-11-16T20:24:16.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640326+0000", 2020-11-16T20:24:16.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022057766999999999 2020-11-16T20:24:16.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:112 3.5s0 3:aa220dd4:::smithi09216568-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:16.385 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:16.385 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.646645209, 2020-11-16T20:24:16.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033427307000000003, 2020-11-16T20:24:16.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:16.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 112 2020-11-16T20:24:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:16.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:16.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002370661 2020-11-16T20:24:16.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642391+0000", 2020-11-16T20:24:16.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3389999999999993e-06 2020-11-16T20:24:16.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642399+0000", 2020-11-16T20:24:16.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.401e-05 2020-11-16T20:24:16.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642413+0000", 2020-11-16T20:24:16.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7309999999999996e-06 2020-11-16T20:24:16.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642420+0000", 2020-11-16T20:24:16.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0634000000000003e-05 2020-11-16T20:24:16.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642480+0000", 2020-11-16T20:24:16.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021235000000000001 2020-11-16T20:24:16.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642693+0000", 2020-11-16T20:24:16.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010232970000000001 2020-11-16T20:24:16.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643716+0000", 2020-11-16T20:24:16.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018482157999999999 2020-11-16T20:24:16.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662198+0000", 2020-11-16T20:24:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011105830000000001 2020-11-16T20:24:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673304+0000", 2020-11-16T20:24:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014329699999999999 2020-11-16T20:24:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673447+0000", 2020-11-16T20:24:16.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033427307000000003 2020-11-16T20:24:16.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.394 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.395 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 31/26 ECSubWrite(tid=7, reqid=client.4444.0:2, at_version=31'3, trim_to=0'0, roll_forward_to=30'1))", 2020-11-16T20:24:16.395 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.647500+0000", 2020-11-16T20:24:16.395 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63916549700000003, 2020-11-16T20:24:16.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023903372999999999, 2020-11-16T20:24:16.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.647500+0000", 2020-11-16T20:24:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.647500+0000", 2020-11-16T20:24:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7890999999999999e-05 2020-11-16T20:24:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.647548+0000", 2020-11-16T20:24:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0234e-05 2020-11-16T20:24:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.647568+0000", 2020-11-16T20:24:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014585407999999999 2020-11-16T20:24:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662153+0000", 2020-11-16T20:24:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0439999999999998e-06 2020-11-16T20:24:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662160+0000", 2020-11-16T20:24:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6156999999999999e-05 2020-11-16T20:24:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662206+0000", 2020-11-16T20:24:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0716999999999999e-05 2020-11-16T20:24:16.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662217+0000", 2020-11-16T20:24:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091443540000000004 2020-11-16T20:24:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.671362+0000", 2020-11-16T20:24:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1567999999999999e-05 2020-11-16T20:24:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.671403+0000", 2020-11-16T20:24:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023903372999999999 2020-11-16T20:24:16.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:113 3.5s0 3:aa220dd4:::smithi09216568-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:16.403 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.634374043, 2020-11-16T20:24:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035489453999999997, 2020-11-16T20:24:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.404 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 113 2020-11-16T20:24:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6527000000000003e-05 2020-11-16T20:24:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652348+0000", 2020-11-16T20:24:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7620000000000001e-06 2020-11-16T20:24:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652356+0000", 2020-11-16T20:24:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2020-11-16T20:24:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652371+0000", 2020-11-16T20:24:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0709999999999997e-06 2020-11-16T20:24:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652378+0000", 2020-11-16T20:24:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2488000000000003e-05 2020-11-16T20:24:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652431+0000", 2020-11-16T20:24:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107941 2020-11-16T20:24:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652539+0000", 2020-11-16T20:24:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021009645 2020-11-16T20:24:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673549+0000", 2020-11-16T20:24:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010424 2020-11-16T20:24:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673653+0000", 2020-11-16T20:24:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000577025 2020-11-16T20:24:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674230+0000", 2020-11-16T20:24:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061432429999999996 2020-11-16T20:24:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.680373+0000", 2020-11-16T20:24:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002934313 2020-11-16T20:24:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.683307+0000", 2020-11-16T20:24:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004473271 2020-11-16T20:24:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.687781+0000", 2020-11-16T20:24:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035489453999999997 2020-11-16T20:24:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:114 3.5s0 3:aa220dd4:::smithi09216568-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63406820600000002, 2020-11-16T20:24:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037793588000000003, 2020-11-16T20:24:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 114 2020-11-16T20:24:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5556000000000003e-05 2020-11-16T20:24:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652633+0000", 2020-11-16T20:24:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2020-11-16T20:24:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652640+0000", 2020-11-16T20:24:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3174e-05 2020-11-16T20:24:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652653+0000", 2020-11-16T20:24:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1580000000000002e-06 2020-11-16T20:24:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652659+0000", 2020-11-16T20:24:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.651e-05 2020-11-16T20:24:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652706+0000", 2020-11-16T20:24:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6873000000000006e-05 2020-11-16T20:24:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652802+0000", 2020-11-16T20:24:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021707194999999999 2020-11-16T20:24:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674510+0000", 2020-11-16T20:24:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114652 2020-11-16T20:24:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674624+0000", 2020-11-16T20:24:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013244752 2020-11-16T20:24:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.687869+0000", 2020-11-16T20:24:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000157813 2020-11-16T20:24:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688027+0000", 2020-11-16T20:24:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023638050000000001 2020-11-16T20:24:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.690391+0000", 2020-11-16T20:24:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037793588000000003 2020-11-16T20:24:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:115 3.5s0 3:ade97634:::smithi09216568-39:head [append 0~638976,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63378579999999995, 2020-11-16T20:24:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028710981, 2020-11-16T20:24:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 115 2020-11-16T20:24:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:16.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046393399999999999 2020-11-16T20:24:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653343+0000", 2020-11-16T20:24:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2337000000000001e-05 2020-11-16T20:24:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653356+0000", 2020-11-16T20:24:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7708000000000001e-05 2020-11-16T20:24:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653373+0000", 2020-11-16T20:24:16.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9990000000000008e-06 2020-11-16T20:24:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653381+0000", 2020-11-16T20:24:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2762000000000001e-05 2020-11-16T20:24:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653434+0000", 2020-11-16T20:24:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000220905 2020-11-16T20:24:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653655+0000", 2020-11-16T20:24:16.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001846164 2020-11-16T20:24:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.655501+0000", 2020-11-16T20:24:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007675594 2020-11-16T20:24:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663177+0000", 2020-11-16T20:24:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018319185000000002 2020-11-16T20:24:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681496+0000", 2020-11-16T20:24:16.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4393000000000002e-05 2020-11-16T20:24:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681590+0000", 2020-11-16T20:24:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028710981 2020-11-16T20:24:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:116 3.5s0 3:ade97634:::smithi09216568-39:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63276617899999998, 2020-11-16T20:24:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035068554000000002, 2020-11-16T20:24:16.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 116 2020-11-16T20:24:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6311999999999999e-05 2020-11-16T20:24:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653955+0000", 2020-11-16T20:24:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9060000000000008e-06 2020-11-16T20:24:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653964+0000", 2020-11-16T20:24:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3818000000000001e-05 2020-11-16T20:24:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653978+0000", 2020-11-16T20:24:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.755e-06 2020-11-16T20:24:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653984+0000", 2020-11-16T20:24:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079596330000000007 2020-11-16T20:24:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.661943+0000", 2020-11-16T20:24:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012344799999999999 2020-11-16T20:24:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662067+0000", 2020-11-16T20:24:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019626787999999999 2020-11-16T20:24:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681694+0000", 2020-11-16T20:24:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107509 2020-11-16T20:24:16.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681801+0000", 2020-11-16T20:24:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000509633 2020-11-16T20:24:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682311+0000", 2020-11-16T20:24:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061688259999999996 2020-11-16T20:24:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688480+0000", 2020-11-16T20:24:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4555999999999995e-05 2020-11-16T20:24:16.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688564+0000", 2020-11-16T20:24:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040337000000000001 2020-11-16T20:24:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688968+0000", 2020-11-16T20:24:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035068554000000002 2020-11-16T20:24:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.447 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:117 3.5s0 3:ade97634:::smithi09216568-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63263549299999999, 2020-11-16T20:24:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037529909, 2020-11-16T20:24:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:16.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 117 2020-11-16T20:24:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:16.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7752e-05 2020-11-16T20:24:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654067+0000", 2020-11-16T20:24:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8970000000000002e-06 2020-11-16T20:24:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654074+0000", 2020-11-16T20:24:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2906e-05 2020-11-16T20:24:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654087+0000", 2020-11-16T20:24:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.181e-06 2020-11-16T20:24:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654092+0000", 2020-11-16T20:24:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080038750000000006 2020-11-16T20:24:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662096+0000", 2020-11-16T20:24:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7354000000000003e-05 2020-11-16T20:24:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662194+0000", 2020-11-16T20:24:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020391761000000001 2020-11-16T20:24:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682585+0000", 2020-11-16T20:24:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114128 2020-11-16T20:24:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:16.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682700+0000", 2020-11-16T20:24:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063062220000000002 2020-11-16T20:24:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689006+0000", 2020-11-16T20:24:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5025000000000007e-05 2020-11-16T20:24:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689091+0000", 2020-11-16T20:24:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024688079999999999 2020-11-16T20:24:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.691560+0000", 2020-11-16T20:24:16.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037529909 2020-11-16T20:24:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 31/26 ECSubWrite(tid=8, reqid=client.4444.0:110, at_version=31'4, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:16.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.62460919199999998, 2020-11-16T20:24:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032919114999999999, 2020-11-16T20:24:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6403999999999999e-05 2020-11-16T20:24:16.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662092+0000", 2020-11-16T20:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2906999999999999e-05 2020-11-16T20:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662115+0000", 2020-11-16T20:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027791066999999999 2020-11-16T20:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689906+0000", 2020-11-16T20:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8580000000000002e-06 2020-11-16T20:24:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689913+0000", 2020-11-16T20:24:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0863999999999999e-05 2020-11-16T20:24:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689974+0000", 2020-11-16T20:24:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4901e-05 2020-11-16T20:24:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689989+0000", 2020-11-16T20:24:16.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049304020000000004 2020-11-16T20:24:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.694919+0000", 2020-11-16T20:24:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5711999999999998e-05 2020-11-16T20:24:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.694975+0000", 2020-11-16T20:24:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032919114999999999 2020-11-16T20:24:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 31/26 ECSubWrite(tid=10, reqid=client.4444.0:136, at_version=31'5, trim_to=0'0, roll_forward_to=31'3))", 2020-11-16T20:24:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.59733339699999999, 2020-11-16T20:24:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042429041000000001, 2020-11-16T20:24:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.467 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050619800000000002 2020-11-16T20:24:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689838+0000", 2020-11-16T20:24:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8632e-05 2020-11-16T20:24:16.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689867+0000", 2020-11-16T20:24:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03374162 2020-11-16T20:24:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723608+0000", 2020-11-16T20:24:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5469999999999999e-06 2020-11-16T20:24:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723616+0000", 2020-11-16T20:24:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8924000000000006e-05 2020-11-16T20:24:16.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723685+0000", 2020-11-16T20:24:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7054e-05 2020-11-16T20:24:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:16.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723702+0000", 2020-11-16T20:24:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080210799999999999 2020-11-16T20:24:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.731723+0000", 2020-11-16T20:24:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7985999999999998e-05 2020-11-16T20:24:16.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.731761+0000", 2020-11-16T20:24:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042429041000000001 2020-11-16T20:24:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.2s1 31/23 ECSubRead(tid=47, to_read={3:47d6be26:::smithi09216568-44:head=0,4096,0}, subchunks={3:47d6be26:::smithi09216568-44:head=[0,1]}, attrs_to_read=))", 2020-11-16T20:24:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.56981931500000005, 2020-11-16T20:24:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027734008000000001, 2020-11-16T20:24:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:16.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8687999999999998e-05 2020-11-16T20:24:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716905+0000", 2020-11-16T20:24:16.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0064999999999999e-05 2020-11-16T20:24:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716915+0000", 2020-11-16T20:24:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026392019999999999 2020-11-16T20:24:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743307+0000", 2020-11-16T20:24:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1610999999999999e-05 2020-11-16T20:24:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743318+0000", 2020-11-16T20:24:16.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4503000000000001e-05 2020-11-16T20:24:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743393+0000", 2020-11-16T20:24:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001187121 2020-11-16T20:24:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.744580+0000", 2020-11-16T20:24:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027734008000000001 2020-11-16T20:24:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:16.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:16.482 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:16.482 INFO:tasks.rados.rados.0.smithi092.stdout:45: finishing write tid 3 to smithi09216568-45 2020-11-16T20:24:16.482 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 45 v 10 (ObjNum 44 snap 0 seq_num 44) dirty exists 2020-11-16T20:24:16.482 INFO:tasks.rados.rados.0.smithi092.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2020-11-16T20:24:16.483 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-11-16T20:24:16.484 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:16.481+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:16.546 INFO:tasks.rados.rados.0.smithi092.stdout:26: done (13 left) 2020-11-16T20:24:16.546 INFO:tasks.rados.rados.0.smithi092.stdout:35: done (12 left) 2020-11-16T20:24:16.546 INFO:tasks.rados.rados.0.smithi092.stdout:42: done (11 left) 2020-11-16T20:24:16.547 INFO:tasks.rados.rados.0.smithi092.stdout:43: done (10 left) 2020-11-16T20:24:16.547 INFO:tasks.rados.rados.0.smithi092.stdout:44: done (9 left) 2020-11-16T20:24:16.547 INFO:tasks.rados.rados.0.smithi092.stdout:45: done (8 left) 2020-11-16T20:24:16.547 INFO:tasks.rados.rados.0.smithi092.stdout:46: done (7 left) 2020-11-16T20:24:16.547 INFO:tasks.rados.rados.0.smithi092.stdout:47: done (6 left) 2020-11-16T20:24:16.547 INFO:tasks.rados.rados.0.smithi092.stdout:48: done (5 left) 2020-11-16T20:24:16.547 INFO:tasks.rados.rados.0.smithi092.stdout:49: done (4 left) 2020-11-16T20:24:16.547 INFO:tasks.rados.rados.0.smithi092.stdout:50: done (3 left) 2020-11-16T20:24:16.548 INFO:tasks.rados.rados.0.smithi092.stdout:51: done (2 left) 2020-11-16T20:24:16.548 INFO:tasks.rados.rados.0.smithi092.stdout:53: done (1 left) 2020-11-16T20:24:16.548 INFO:tasks.rados.rados.0.smithi092.stdout:54: done (0 left) 2020-11-16T20:24:16.548 INFO:tasks.rados.rados.0.smithi092.stdout:55: rmattr oid 35 current snap is 1 2020-11-16T20:24:16.548 INFO:tasks.rados.rados.0.smithi092.stdout:55: done (0 left) 2020-11-16T20:24:16.548 INFO:tasks.rados.rados.0.smithi092.stdout:56: rmattr oid 21 current snap is 1 2020-11-16T20:24:16.548 INFO:tasks.rados.rados.0.smithi092.stdout:56: done (0 left) 2020-11-16T20:24:16.549 INFO:tasks.rados.rados.0.smithi092.stdout:57: setattr oid 20 current snap is 1 2020-11-16T20:24:16.549 INFO:tasks.rados.rados.0.smithi092.stdout:58: rollback oid 35 current snap is 1 2020-11-16T20:24:16.549 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 35 to 0 2020-11-16T20:24:16.549 INFO:tasks.rados.rados.0.smithi092.stdout:59: delete oid 37 current snap is 1 2020-11-16T20:24:16.563 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2020-11-16T20:24:16.563 INFO:tasks.rados.rados.0.smithi092.stdout:58: finishing rollback tid 0 to smithi09216568-35 2020-11-16T20:24:16.563 INFO:tasks.rados.rados.0.smithi092.stdout:57: done (2 left) 2020-11-16T20:24:16.564 INFO:tasks.rados.rados.0.smithi092.stdout:59: done (1 left) 2020-11-16T20:24:16.564 INFO:tasks.rados.rados.0.smithi092.stdout:60: rmattr oid 6 current snap is 1 2020-11-16T20:24:16.564 INFO:tasks.rados.rados.0.smithi092.stdout:60: done (1 left) 2020-11-16T20:24:16.564 INFO:tasks.rados.rados.0.smithi092.stdout:61: setattr oid 25 current snap is 1 2020-11-16T20:24:16.565 INFO:tasks.rados.rados.0.smithi092.stdout:58: finishing rollback tid 1 to smithi09216568-35 2020-11-16T20:24:16.565 INFO:tasks.rados.rados.0.smithi092.stdout:62: copy_from oid 23 from oid 7 current snap is 1 2020-11-16T20:24:16.565 INFO:tasks.rados.rados.0.smithi092.stdout:63: snap_create 2020-11-16T20:24:16.565 INFO:tasks.rados.rados.0.smithi092.stdout:58: finishing rollback tid 2 to smithi09216568-35 2020-11-16T20:24:16.565 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 35 v 15 (ObjNum 34 snap 0 seq_num 34) dirty exists 2020-11-16T20:24:16.569 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 25 v 10 (ObjNum 24 snap 0 seq_num 24) dirty exists 2020-11-16T20:24:16.577 INFO:tasks.rados.rados.0.smithi092.stdout:62: finishing copy_from to smithi09216568-23 2020-11-16T20:24:16.577 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 4 (ObjNum 6 snap 0 seq_num 6) dirty exists 2020-11-16T20:24:16.578 INFO:tasks.rados.rados.0.smithi092.stdout:62: finishing copy_from racing read to smithi09216568-23 2020-11-16T20:24:16.584 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:16.580+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:16.662 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-11-16T20:24:16.685 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:16.684+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:16.786 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:16.785+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:16.828 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:16.829 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:16.838 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-11-16T20:24:16.887 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:16.884+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:21.023 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T20:24:21.024 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T20:24:21.024 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-11-16T20:24:21.024 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2020-11-16T20:24:21.029 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:14 3.9s0 3:910fffa8:::smithi09216568-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-11-16T20:24:21.030 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.471108648, 2020-11-16T20:24:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033104548999999997, 2020-11-16T20:24:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 14 2020-11-16T20:24:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:21.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8910999999999999e-05 2020-11-16T20:24:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532325+0000", 2020-11-16T20:24:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2323e-05 2020-11-16T20:24:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532347+0000", 2020-11-16T20:24:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7090999999999999e-05 2020-11-16T20:24:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532374+0000", 2020-11-16T20:24:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5399999999999998e-06 2020-11-16T20:24:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532382+0000", 2020-11-16T20:24:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077685499999999999 2020-11-16T20:24:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540150+0000", 2020-11-16T20:24:21.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119175 2020-11-16T20:24:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540269+0000", 2020-11-16T20:24:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011674891 2020-11-16T20:24:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551944+0000", 2020-11-16T20:24:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010398199999999999 2020-11-16T20:24:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.552048+0000", 2020-11-16T20:24:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077962600000000002 2020-11-16T20:24:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.559845+0000", 2020-11-16T20:24:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028470140000000001 2020-11-16T20:24:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562692+0000", 2020-11-16T20:24:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002587677 2020-11-16T20:24:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565279+0000", 2020-11-16T20:24:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1135000000000001e-05 2020-11-16T20:24:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565370+0000", 2020-11-16T20:24:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033104548999999997 2020-11-16T20:24:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:15 3.9s0 3:910fffa8:::smithi09216568-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e30)", 2020-11-16T20:24:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4709410510000001, 2020-11-16T20:24:21.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037521408999999999, 2020-11-16T20:24:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15 2020-11-16T20:24:21.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9020000000000001e-05 2020-11-16T20:24:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532493+0000", 2020-11-16T20:24:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0396e-05 2020-11-16T20:24:21.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532503+0000", 2020-11-16T20:24:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9660000000000002e-05 2020-11-16T20:24:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532523+0000", 2020-11-16T20:24:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8280000000000003e-06 2020-11-16T20:24:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532530+0000", 2020-11-16T20:24:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077981070000000003 2020-11-16T20:24:21.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540329+0000", 2020-11-16T20:24:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8363000000000007e-05 2020-11-16T20:24:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540427+0000", 2020-11-16T20:24:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019650447000000001 2020-11-16T20:24:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560077+0000", 2020-11-16T20:24:21.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2565999999999994e-05 2020-11-16T20:24:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560170+0000", 2020-11-16T20:24:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052459309999999997 2020-11-16T20:24:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565416+0000", 2020-11-16T20:24:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1088999999999995e-05 2020-11-16T20:24:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565507+0000", 2020-11-16T20:24:21.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044480020000000004 2020-11-16T20:24:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.569955+0000", 2020-11-16T20:24:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037521408999999999 2020-11-16T20:24:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:23 3.ds0 3:b099cead:::smithi09216568-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-11-16T20:24:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.470496732, 2020-11-16T20:24:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025302294999999999, 2020-11-16T20:24:21.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23 2020-11-16T20:24:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5377999999999997e-05 2020-11-16T20:24:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532933+0000", 2020-11-16T20:24:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9703000000000001e-05 2020-11-16T20:24:21.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532953+0000", 2020-11-16T20:24:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2229e-05 2020-11-16T20:24:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532975+0000", 2020-11-16T20:24:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041310599999999998 2020-11-16T20:24:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537106+0000", 2020-11-16T20:24:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055730839999999999 2020-11-16T20:24:21.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542679+0000", 2020-11-16T20:24:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013964700000000001 2020-11-16T20:24:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542819+0000", 2020-11-16T20:24:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050541550000000003 2020-11-16T20:24:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.547873+0000", 2020-11-16T20:24:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253999999999993e-05 2020-11-16T20:24:21.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.547934+0000", 2020-11-16T20:24:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000519886 2020-11-16T20:24:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548454+0000", 2020-11-16T20:24:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066483360000000003 2020-11-16T20:24:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.555103+0000", 2020-11-16T20:24:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026224410000000001 2020-11-16T20:24:21.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557725+0000", 2020-11-16T20:24:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045512200000000002 2020-11-16T20:24:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558180+0000", 2020-11-16T20:24:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025302294999999999 2020-11-16T20:24:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:24 3.ds0 3:b099cead:::smithi09216568-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e30)", 2020-11-16T20:24:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.466202794, 2020-11-16T20:24:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025939752999999999, 2020-11-16T20:24:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 24 2020-11-16T20:24:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:21.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8302e-05 2020-11-16T20:24:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537210+0000", 2020-11-16T20:24:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5029999999999997e-06 2020-11-16T20:24:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537219+0000", 2020-11-16T20:24:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5136e-05 2020-11-16T20:24:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537234+0000", 2020-11-16T20:24:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2160000000000003e-06 2020-11-16T20:24:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537240+0000", 2020-11-16T20:24:21.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056701290000000003 2020-11-16T20:24:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542910+0000", 2020-11-16T20:24:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011685599999999999 2020-11-16T20:24:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543027+0000", 2020-11-16T20:24:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056655689999999996 2020-11-16T20:24:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548692+0000", 2020-11-16T20:24:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9920000000000003e-05 2020-11-16T20:24:21.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548742+0000", 2020-11-16T20:24:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094867259999999991 2020-11-16T20:24:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558229+0000", 2020-11-16T20:24:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8435000000000003e-05 2020-11-16T20:24:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558318+0000", 2020-11-16T20:24:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047939610000000002 2020-11-16T20:24:21.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563111+0000", 2020-11-16T20:24:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025939752999999999 2020-11-16T20:24:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:32 3.9s0 3:94376c81:::smithi09216568-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.463812332, 2020-11-16T20:24:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026430479999999999, 2020-11-16T20:24:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32 2020-11-16T20:24:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:21.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5041999999999999e-05 2020-11-16T20:24:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539617+0000", 2020-11-16T20:24:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2020-11-16T20:24:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539624+0000", 2020-11-16T20:24:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2020-11-16T20:24:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539636+0000", 2020-11-16T20:24:21.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.075e-06 2020-11-16T20:24:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539643+0000", 2020-11-16T20:24:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039536490000000001 2020-11-16T20:24:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543597+0000", 2020-11-16T20:24:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112988 2020-11-16T20:24:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543710+0000", 2020-11-16T20:24:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016926085 2020-11-16T20:24:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560636+0000", 2020-11-16T20:24:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8800000000000004e-05 2020-11-16T20:24:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560725+0000", 2020-11-16T20:24:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00080697699999999995 2020-11-16T20:24:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.561532+0000", 2020-11-16T20:24:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042868910000000001 2020-11-16T20:24:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565818+0000", 2020-11-16T20:24:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5162999999999998e-05 2020-11-16T20:24:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565904+0000", 2020-11-16T20:24:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9067999999999995e-05 2020-11-16T20:24:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565993+0000", 2020-11-16T20:24:21.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026430479999999999 2020-11-16T20:24:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:33 3.9s0 3:94376c81:::smithi09216568-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:21.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.463690538, 2020-11-16T20:24:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030497335, 2020-11-16T20:24:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33 2020-11-16T20:24:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8398000000000001e-05 2020-11-16T20:24:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539722+0000", 2020-11-16T20:24:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.993e-06 2020-11-16T20:24:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539728+0000", 2020-11-16T20:24:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2140000000000001e-05 2020-11-16T20:24:21.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539741+0000", 2020-11-16T20:24:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9979999999999997e-06 2020-11-16T20:24:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539745+0000", 2020-11-16T20:24:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004070204 2020-11-16T20:24:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543815+0000", 2020-11-16T20:24:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535999999999999 2020-11-16T20:24:21.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543920+0000", 2020-11-16T20:24:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018410834000000001 2020-11-16T20:24:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562331+0000", 2020-11-16T20:24:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4544999999999997e-05 2020-11-16T20:24:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562425+0000", 2020-11-16T20:24:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035976329999999998 2020-11-16T20:24:21.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.566023+0000", 2020-11-16T20:24:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2955999999999996e-05 2020-11-16T20:24:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.566116+0000", 2020-11-16T20:24:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040652739999999998 2020-11-16T20:24:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.570181+0000", 2020-11-16T20:24:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030497335 2020-11-16T20:24:21.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:35 3.9s0 3:9cc22e43:::smithi09216568-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4625179909999999, 2020-11-16T20:24:21.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11108446600000001, 2020-11-16T20:24:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 35 2020-11-16T20:24:21.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4415e-05 2020-11-16T20:24:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540911+0000", 2020-11-16T20:24:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.533e-06 2020-11-16T20:24:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540917+0000", 2020-11-16T20:24:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-05 2020-11-16T20:24:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540930+0000", 2020-11-16T20:24:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2769999999999999e-06 2020-11-16T20:24:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540934+0000", 2020-11-16T20:24:21.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010586527 2020-11-16T20:24:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551521+0000", 2020-11-16T20:24:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012027399999999999 2020-11-16T20:24:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551641+0000", 2020-11-16T20:24:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011083160999999999 2020-11-16T20:24:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562724+0000", 2020-11-16T20:24:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4109000000000004e-05 2020-11-16T20:24:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562808+0000", 2020-11-16T20:24:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000459751 2020-11-16T20:24:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563268+0000", 2020-11-16T20:24:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060973590000000001 2020-11-16T20:24:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.569365+0000", 2020-11-16T20:24:21.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082458486999999997 2020-11-16T20:24:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651824+0000", 2020-11-16T20:24:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117173 2020-11-16T20:24:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651941+0000", 2020-11-16T20:24:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11108446600000001 2020-11-16T20:24:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:36 3.9s0 3:9cc22e43:::smithi09216568-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4623989180000001, 2020-11-16T20:24:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.116868257, 2020-11-16T20:24:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36 2020-11-16T20:24:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9727000000000001e-05 2020-11-16T20:24:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541015+0000", 2020-11-16T20:24:21.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1739999999999997e-06 2020-11-16T20:24:21.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541022+0000", 2020-11-16T20:24:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1854e-05 2020-11-16T20:24:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541033+0000", 2020-11-16T20:24:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.799e-06 2020-11-16T20:24:21.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541037+0000", 2020-11-16T20:24:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019161628999999999 2020-11-16T20:24:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560199+0000", 2020-11-16T20:24:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102256 2020-11-16T20:24:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560301+0000", 2020-11-16T20:24:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032318479999999998 2020-11-16T20:24:21.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563533+0000", 2020-11-16T20:24:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2615999999999996e-05 2020-11-16T20:24:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563626+0000", 2020-11-16T20:24:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088372749 2020-11-16T20:24:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651998+0000", 2020-11-16T20:24:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113002 2020-11-16T20:24:21.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652111+0000", 2020-11-16T20:24:21.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057326030000000002 2020-11-16T20:24:21.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.657844+0000", 2020-11-16T20:24:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.116868257 2020-11-16T20:24:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.120 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=10, last_complete=31'6, committed=1, applied=1))", 2020-11-16T20:24:21.121 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:21.121 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4377571419999999, 2020-11-16T20:24:21.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086340027, 2020-11-16T20:24:21.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:21.121 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:21.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:21.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6184999999999999e-05 2020-11-16T20:24:21.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565664+0000", 2020-11-16T20:24:21.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7180000000000002e-06 2020-11-16T20:24:21.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565672+0000", 2020-11-16T20:24:21.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085916762999999993 2020-11-16T20:24:21.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651589+0000", 2020-11-16T20:24:21.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5119999999999997e-06 2020-11-16T20:24:21.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651596+0000", 2020-11-16T20:24:21.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3547000000000001e-05 2020-11-16T20:24:21.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651649+0000", 2020-11-16T20:24:21.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030830199999999999 2020-11-16T20:24:21.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651957+0000", 2020-11-16T20:24:21.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086340027 2020-11-16T20:24:21.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.127 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:85 3.9s0 3:9bc9395c:::smithi09216568-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:21.127 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:21.127 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.399658351, 2020-11-16T20:24:21.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048890797999999999, 2020-11-16T20:24:21.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 85 2020-11-16T20:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:21.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033801700000000003 2020-11-16T20:24:21.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604054+0000", 2020-11-16T20:24:21.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1904e-05 2020-11-16T20:24:21.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604066+0000", 2020-11-16T20:24:21.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2374e-05 2020-11-16T20:24:21.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604088+0000", 2020-11-16T20:24:21.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0937e-05 2020-11-16T20:24:21.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604099+0000", 2020-11-16T20:24:21.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6842999999999996e-05 2020-11-16T20:24:21.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604176+0000", 2020-11-16T20:24:21.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180261 2020-11-16T20:24:21.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604357+0000", 2020-11-16T20:24:21.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012281449999999999 2020-11-16T20:24:21.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605585+0000", 2020-11-16T20:24:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074810600000000003 2020-11-16T20:24:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613066+0000", 2020-11-16T20:24:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039419787999999997 2020-11-16T20:24:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652486+0000", 2020-11-16T20:24:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121469 2020-11-16T20:24:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652607+0000", 2020-11-16T20:24:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048890797999999999 2020-11-16T20:24:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:86 3.9s0 3:9bc9395c:::smithi09216568-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.399216671, 2020-11-16T20:24:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071173710000000001, 2020-11-16T20:24:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 86 2020-11-16T20:24:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7161999999999999e-05 2020-11-16T20:24:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604215+0000", 2020-11-16T20:24:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3989999999999998e-06 2020-11-16T20:24:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604222+0000", 2020-11-16T20:24:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3733999999999999e-05 2020-11-16T20:24:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604236+0000", 2020-11-16T20:24:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3090000000000003e-06 2020-11-16T20:24:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604241+0000", 2020-11-16T20:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074329820000000003 2020-11-16T20:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611674+0000", 2020-11-16T20:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117886 2020-11-16T20:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611792+0000", 2020-11-16T20:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040893269000000003 2020-11-16T20:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652686+0000", 2020-11-16T20:24:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9841999999999998e-05 2020-11-16T20:24:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652775+0000", 2020-11-16T20:24:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052300899999999995 2020-11-16T20:24:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653298+0000", 2020-11-16T20:24:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089837800000000002 2020-11-16T20:24:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654197+0000", 2020-11-16T20:24:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021031319999999999 2020-11-16T20:24:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675228+0000", 2020-11-16T20:24:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010342 2020-11-16T20:24:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675332+0000", 2020-11-16T20:24:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071173710000000001 2020-11-16T20:24:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:87 3.9s0 3:9bc9395c:::smithi09216568-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3990858429999999, 2020-11-16T20:24:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072859264000000007, 2020-11-16T20:24:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 87 2020-11-16T20:24:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1205000000000003e-05 2020-11-16T20:24:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604330+0000", 2020-11-16T20:24:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7689999999999999e-06 2020-11-16T20:24:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604337+0000", 2020-11-16T20:24:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3181e-05 2020-11-16T20:24:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604350+0000", 2020-11-16T20:24:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1000000000000003e-06 2020-11-16T20:24:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604355+0000", 2020-11-16T20:24:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074779859999999998 2020-11-16T20:24:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611833+0000", 2020-11-16T20:24:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087081400000000003 2020-11-16T20:24:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612704+0000", 2020-11-16T20:24:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040880316 2020-11-16T20:24:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653584+0000", 2020-11-16T20:24:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5697000000000003e-05 2020-11-16T20:24:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653670+0000", 2020-11-16T20:24:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021708466999999999 2020-11-16T20:24:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675378+0000", 2020-11-16T20:24:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7913999999999998e-05 2020-11-16T20:24:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675476+0000", 2020-11-16T20:24:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016718150000000001 2020-11-16T20:24:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.677148+0000", 2020-11-16T20:24:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072859264000000007 2020-11-16T20:24:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:103 3.9s0 3:98ee58dd:::smithi09216568-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3774474080000001, 2020-11-16T20:24:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037777976999999997, 2020-11-16T20:24:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 103 2020-11-16T20:24:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000376745 2020-11-16T20:24:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626304+0000", 2020-11-16T20:24:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0266e-05 2020-11-16T20:24:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626314+0000", 2020-11-16T20:24:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327e-05 2020-11-16T20:24:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626334+0000", 2020-11-16T20:24:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0482999999999999e-05 2020-11-16T20:24:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626345+0000", 2020-11-16T20:24:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7775999999999996e-05 2020-11-16T20:24:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626413+0000", 2020-11-16T20:24:21.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020824099999999999 2020-11-16T20:24:21.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626621+0000", 2020-11-16T20:24:21.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097088399999999998 2020-11-16T20:24:21.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.627592+0000", 2020-11-16T20:24:21.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074737739999999999 2020-11-16T20:24:21.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635066+0000", 2020-11-16T20:24:21.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028500588 2020-11-16T20:24:21.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663566+0000", 2020-11-16T20:24:21.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013889300000000001 2020-11-16T20:24:21.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663705+0000", 2020-11-16T20:24:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037777976999999997 2020-11-16T20:24:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:104 3.9s0 3:98ee58dd:::smithi09216568-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3769737179999999, 2020-11-16T20:24:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109118772, 2020-11-16T20:24:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104 2020-11-16T20:24:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4968999999999997e-05 2020-11-16T20:24:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626436+0000", 2020-11-16T20:24:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.33e-06 2020-11-16T20:24:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626440+0000", 2020-11-16T20:24:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1780000000000003e-06 2020-11-16T20:24:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626448+0000", 2020-11-16T20:24:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5559999999999998e-06 2020-11-16T20:24:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626454+0000", 2020-11-16T20:24:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082374119999999995 2020-11-16T20:24:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634691+0000", 2020-11-16T20:24:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6558999999999995e-05 2020-11-16T20:24:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634788+0000", 2020-11-16T20:24:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029028163999999999 2020-11-16T20:24:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663816+0000", 2020-11-16T20:24:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9767999999999998e-05 2020-11-16T20:24:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663906+0000", 2020-11-16T20:24:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088046700000000001 2020-11-16T20:24:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.664786+0000", 2020-11-16T20:24:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001328255 2020-11-16T20:24:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.666114+0000", 2020-11-16T20:24:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069334706999999995 2020-11-16T20:24:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735449+0000", 2020-11-16T20:24:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0407000000000005e-05 2020-11-16T20:24:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735520+0000", 2020-11-16T20:24:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109118772 2020-11-16T20:24:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:105 3.9s0 3:98ee58dd:::smithi09216568-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3768745280000001, 2020-11-16T20:24:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11144889600000001, 2020-11-16T20:24:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2020-11-16T20:24:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0299000000000002e-05 2020-11-16T20:24:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626540+0000", 2020-11-16T20:24:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3050000000000004e-06 2020-11-16T20:24:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626548+0000", 2020-11-16T20:24:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.878e-05 2020-11-16T20:24:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626566+0000", 2020-11-16T20:24:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3839999999999996e-06 2020-11-16T20:24:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626572+0000", 2020-11-16T20:24:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082488189999999992 2020-11-16T20:24:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634821+0000", 2020-11-16T20:24:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4552e-05 2020-11-16T20:24:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634905+0000", 2020-11-16T20:24:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030415658000000002 2020-11-16T20:24:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.665321+0000", 2020-11-16T20:24:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0865000000000005e-05 2020-11-16T20:24:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.665412+0000", 2020-11-16T20:24:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.070142494 2020-11-16T20:24:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735554+0000", 2020-11-16T20:24:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9005000000000007e-05 2020-11-16T20:24:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735633+0000", 2020-11-16T20:24:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023157350000000002 2020-11-16T20:24:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.737949+0000", 2020-11-16T20:24:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11144889600000001 2020-11-16T20:24:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:108 3.as0 3:5d07813d:::smithi09216568-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3724462909999999, 2020-11-16T20:24:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024149034999999999, 2020-11-16T20:24:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 108 2020-11-16T20:24:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4607999999999997e-05 2020-11-16T20:24:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630973+0000", 2020-11-16T20:24:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4680000000000003e-06 2020-11-16T20:24:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630980+0000", 2020-11-16T20:24:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019917399999999999 2020-11-16T20:24:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631179+0000", 2020-11-16T20:24:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4740000000000002e-06 2020-11-16T20:24:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631186+0000", 2020-11-16T20:24:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010229520000000001 2020-11-16T20:24:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641415+0000", 2020-11-16T20:24:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3351999999999997e-05 2020-11-16T20:24:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641499+0000", 2020-11-16T20:24:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074322040000000004 2020-11-16T20:24:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.648931+0000", 2020-11-16T20:24:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.166e-05 2020-11-16T20:24:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.649023+0000", 2020-11-16T20:24:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039121529999999998 2020-11-16T20:24:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652935+0000", 2020-11-16T20:24:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140617 2020-11-16T20:24:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653075+0000", 2020-11-16T20:24:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020018050000000002 2020-11-16T20:24:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.655077+0000", 2020-11-16T20:24:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024149034999999999 2020-11-16T20:24:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=24, last_complete=31'10, committed=1, applied=1))", 2020-11-16T20:24:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3283895509999999, 2020-11-16T20:24:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060547953000000002, 2020-11-16T20:24:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6891000000000003e-05 2020-11-16T20:24:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675022+0000", 2020-11-16T20:24:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3489999999999998e-06 2020-11-16T20:24:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675031+0000", 2020-11-16T20:24:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060268058999999999 2020-11-16T20:24:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735299+0000", 2020-11-16T20:24:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7539999999999995e-06 2020-11-16T20:24:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735308+0000", 2020-11-16T20:24:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7015999999999998e-05 2020-11-16T20:24:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735355+0000", 2020-11-16T20:24:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017788400000000001 2020-11-16T20:24:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735533+0000", 2020-11-16T20:24:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060547953000000002 2020-11-16T20:24:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:153 3.9s0 3:94376c81:::smithi09216568-11:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.292584797, 2020-11-16T20:24:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056909992999999999, 2020-11-16T20:24:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 153 2020-11-16T20:24:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6903999999999998e-05 2020-11-16T20:24:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710827+0000", 2020-11-16T20:24:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0011e-05 2020-11-16T20:24:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710837+0000", 2020-11-16T20:24:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4769e-05 2020-11-16T20:24:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710851+0000", 2020-11-16T20:24:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1129999999999997e-06 2020-11-16T20:24:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710858+0000", 2020-11-16T20:24:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6598999999999998e-05 2020-11-16T20:24:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710914+0000", 2020-11-16T20:24:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110771 2020-11-16T20:24:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711025+0000", 2020-11-16T20:24:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011061122 2020-11-16T20:24:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.722086+0000", 2020-11-16T20:24:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055101880000000001 2020-11-16T20:24:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.727596+0000", 2020-11-16T20:24:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039941002000000003 2020-11-16T20:24:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767537+0000", 2020-11-16T20:24:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000162514 2020-11-16T20:24:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767700+0000", 2020-11-16T20:24:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056909992999999999 2020-11-16T20:24:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:154 3.9s0 3:94376c81:::smithi09216568-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.292465333, 2020-11-16T20:24:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058295893000000001, 2020-11-16T20:24:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2020-11-16T20:24:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7756999999999998e-05 2020-11-16T20:24:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710947+0000", 2020-11-16T20:24:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7429999999999996e-06 2020-11-16T20:24:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710954+0000", 2020-11-16T20:24:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2998e-05 2020-11-16T20:24:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710967+0000", 2020-11-16T20:24:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8250000000000004e-06 2020-11-16T20:24:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710972+0000", 2020-11-16T20:24:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022201499999999999 2020-11-16T20:24:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711194+0000", 2020-11-16T20:24:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1608e-05 2020-11-16T20:24:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2020-11-16T20:24:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711225+0000", 2020-11-16T20:24:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012795857000000001 2020-11-16T20:24:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724021+0000", 2020-11-16T20:24:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7220999999999998e-05 2020-11-16T20:24:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724098+0000", 2020-11-16T20:24:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044522785000000002 2020-11-16T20:24:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768621+0000", 2020-11-16T20:24:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026034400000000003 2020-11-16T20:24:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768881+0000", 2020-11-16T20:24:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032373999999999999 2020-11-16T20:24:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.769205+0000", 2020-11-16T20:24:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058295893000000001 2020-11-16T20:24:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=26, last_complete=31'11, committed=1, applied=1))", 2020-11-16T20:24:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.279025632, 2020-11-16T20:24:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044143454999999998, 2020-11-16T20:24:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7308999999999999e-05 2020-11-16T20:24:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724376+0000", 2020-11-16T20:24:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9070000000000004e-06 2020-11-16T20:24:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724382+0000", 2020-11-16T20:24:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042942860999999999 2020-11-16T20:24:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:21.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767325+0000", 2020-11-16T20:24:21.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2480000000000001e-05 2020-11-16T20:24:21.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:21.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767337+0000", 2020-11-16T20:24:21.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.177e-05 2020-11-16T20:24:21.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:21.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767399+0000", 2020-11-16T20:24:21.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010931280000000001 2020-11-16T20:24:21.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:21.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:21.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:21.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768492+0000", 2020-11-16T20:24:21.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044143454999999998 2020-11-16T20:24:21.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.237 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:21.237 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:21.238 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:21.238 INFO:tasks.rados.rados.0.smithi092.stdout:58: done (3 left) 2020-11-16T20:24:21.238 INFO:tasks.rados.rados.0.smithi092.stdout:61: done (2 left) 2020-11-16T20:24:21.238 INFO:tasks.rados.rados.0.smithi092.stdout:62: done (1 left) 2020-11-16T20:24:21.238 INFO:tasks.rados.rados.0.smithi092.stdout:63: done (0 left) 2020-11-16T20:24:21.238 INFO:tasks.rados.rados.0.smithi092.stdout:64: setattr oid 44 current snap is 2 2020-11-16T20:24:21.239 INFO:tasks.rados.rados.0.smithi092.stdout:65: rollback oid 5 current snap is 2 2020-11-16T20:24:21.239 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 5 to 1 2020-11-16T20:24:21.239 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 39 current snap is 2 2020-11-16T20:24:21.239 INFO:tasks.rados.rados.0.smithi092.stdout:66: seq_num 51 ranges {638976=458752,1097728=401408} 2020-11-16T20:24:21.239 INFO:tasks.rados.rados.0.smithi092.stdout:66: writing smithi09216568-39 from 638976 to 1097728 tid 1 2020-11-16T20:24:21.239 INFO:tasks.rados.rados.0.smithi092.stdout:66: writing smithi09216568-39 from 1097728 to 1499136 tid 2 2020-11-16T20:24:21.239 INFO:tasks.rados.rados.0.smithi092.stdout:67: read oid 14 snap 1 2020-11-16T20:24:21.239 INFO:tasks.rados.rados.0.smithi092.stdout:67: expect (ObjNum 13 snap 0 seq_num 13) 2020-11-16T20:24:21.240 INFO:tasks.rados.rados.0.smithi092.stdout:68: setattr oid 31 current snap is 2 2020-11-16T20:24:21.240 INFO:tasks.rados.rados.0.smithi092.stdout:65: finishing rollback tid 0 to smithi09216568-5 2020-11-16T20:24:21.240 INFO:tasks.rados.rados.0.smithi092.stdout:65: finishing rollback tid 1 to smithi09216568-5 2020-11-16T20:24:21.240 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 44 v 8 (ObjNum 43 snap 0 seq_num 43) dirty exists 2020-11-16T20:24:21.240 INFO:tasks.rados.rados.0.smithi092.stdout:65: finishing rollback tid 2 to smithi09216568-5 2020-11-16T20:24:21.240 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 19 (ObjNum 4 snap 0 seq_num 4) dirty exists 2020-11-16T20:24:21.240 INFO:tasks.rados.rados.0.smithi092.stdout:64: done (4 left) 2020-11-16T20:24:21.241 INFO:tasks.rados.rados.0.smithi092.stdout:65: done (3 left) 2020-11-16T20:24:21.241 INFO:tasks.rados.rados.0.smithi092.stdout:69: delete oid 8 current snap is 2 2020-11-16T20:24:21.241 INFO:tasks.rados.rados.0.smithi092.stdout:67: expect (ObjNum 13 snap 0 seq_num 13) 2020-11-16T20:24:21.241 INFO:tasks.rados.rados.0.smithi092.stdout:67: done (3 left) 2020-11-16T20:24:21.241 INFO:tasks.rados.rados.0.smithi092.stdout:69: done (2 left) 2020-11-16T20:24:21.241 INFO:tasks.rados.rados.0.smithi092.stdout:70: delete oid 15 current snap is 2 2020-11-16T20:24:21.241 INFO:tasks.rados.rados.0.smithi092.stdout:66: finishing write tid 1 to smithi09216568-39 2020-11-16T20:24:21.241 INFO:tasks.rados.rados.0.smithi092.stdout:66: finishing write tid 2 to smithi09216568-39 2020-11-16T20:24:21.242 INFO:tasks.rados.rados.0.smithi092.stdout:70: done (2 left) 2020-11-16T20:24:21.242 INFO:tasks.rados.rados.0.smithi092.stdout:71: copy_from oid 48 from oid 16 current snap is 2 2020-11-16T20:24:21.242 INFO:tasks.rados.rados.0.smithi092.stdout:72: snap_create 2020-11-16T20:24:21.242 INFO:tasks.rados.rados.0.smithi092.stdout:66: finishing write tid 3 to smithi09216568-39 2020-11-16T20:24:21.242 INFO:tasks.rados.rados.0.smithi092.stdout:66: finishing write tid 4 to smithi09216568-39 2020-11-16T20:24:21.242 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 39 v 10 (ObjNum 51 snap 2 seq_num 51) dirty exists 2020-11-16T20:24:21.242 INFO:tasks.rados.rados.0.smithi092.stdout:66: left oid 39 (ObjNum 51 snap 2 seq_num 51) 2020-11-16T20:24:21.243 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2020-11-16T20:24:21.243 INFO:tasks.rados.rados.0.smithi092.stdout:71: finishing copy_from to smithi09216568-48 2020-11-16T20:24:21.243 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 48 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2020-11-16T20:24:21.243 INFO:tasks.rados.rados.0.smithi092.stdout:71: finishing copy_from racing read to smithi09216568-48 2020-11-16T20:24:21.243 INFO:tasks.rados.rados.0.smithi092.stdout:66: done (3 left) 2020-11-16T20:24:21.243 INFO:tasks.rados.rados.0.smithi092.stdout:68: done (2 left) 2020-11-16T20:24:21.243 INFO:tasks.rados.rados.0.smithi092.stdout:71: done (1 left) 2020-11-16T20:24:21.243 INFO:tasks.rados.rados.0.smithi092.stdout:72: done (0 left) 2020-11-16T20:24:21.244 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 20 current snap is 3 2020-11-16T20:24:21.244 INFO:tasks.rados.rados.0.smithi092.stdout:73: seq_num 54 ranges {319488=507904,827392=401408} 2020-11-16T20:24:21.244 INFO:tasks.rados.rados.0.smithi092.stdout:73: writing smithi09216568-20 from 319488 to 827392 tid 1 2020-11-16T20:24:21.244 INFO:tasks.rados.rados.0.smithi092.stdout:73: writing smithi09216568-20 from 827392 to 1228800 tid 2 2020-11-16T20:24:21.244 INFO:tasks.rados.rados.0.smithi092.stdout:74: setattr oid 48 current snap is 3 2020-11-16T20:24:21.244 INFO:tasks.rados.rados.0.smithi092.stdout:75: delete oid 23 current snap is 3 2020-11-16T20:24:21.244 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 48 v 17 (ObjNum 15 snap 0 seq_num 15) dirty exists 2020-11-16T20:24:21.244 INFO:tasks.rados.rados.0.smithi092.stdout:74: done (2 left) 2020-11-16T20:24:21.245 INFO:tasks.rados.rados.0.smithi092.stdout:75: done (1 left) 2020-11-16T20:24:21.245 INFO:tasks.rados.rados.0.smithi092.stdout:76: rollback oid 8 current snap is 3 2020-11-16T20:24:21.245 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 8 to 1 2020-11-16T20:24:21.245 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 43 current snap is 3 2020-11-16T20:24:21.245 INFO:tasks.rados.rados.0.smithi092.stdout:77: seq_num 56 ranges {712704=540672,1253376=401408} 2020-11-16T20:24:21.245 INFO:tasks.rados.rados.0.smithi092.stdout:77: writing smithi09216568-43 from 712704 to 1253376 tid 1 2020-11-16T20:24:21.245 INFO:tasks.rados.rados.0.smithi092.stdout:77: writing smithi09216568-43 from 1253376 to 1654784 tid 2 2020-11-16T20:24:21.246 INFO:tasks.rados.rados.0.smithi092.stdout:78: copy_from oid 10 from oid 35 current snap is 3 2020-11-16T20:24:21.246 INFO:tasks.rados.rados.0.smithi092.stdout:76: finishing rollback tid 1 to smithi09216568-8 2020-11-16T20:24:21.246 INFO:tasks.rados.rados.0.smithi092.stdout:76: finishing rollback tid 2 to smithi09216568-8 2020-11-16T20:24:21.246 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 8 v 8 (ObjNum 7 snap 0 seq_num 7) dirty exists 2020-11-16T20:24:21.246 INFO:tasks.rados.rados.0.smithi092.stdout:76: done (3 left) 2020-11-16T20:24:21.246 INFO:tasks.rados.rados.0.smithi092.stdout:79: snap_remove snap 0 2020-11-16T20:24:21.246 INFO:tasks.rados.rados.0.smithi092.stdout:79: done (3 left) 2020-11-16T20:24:21.246 INFO:tasks.rados.rados.0.smithi092.stdout:80: read oid 46 snap 1 2020-11-16T20:24:21.247 INFO:tasks.rados.rados.0.smithi092.stdout:80: expect (ObjNum 45 snap 0 seq_num 45) 2020-11-16T20:24:21.247 INFO:tasks.rados.rados.0.smithi092.stdout:73: finishing write tid 1 to smithi09216568-20 2020-11-16T20:24:21.247 INFO:tasks.rados.rados.0.smithi092.stdout:73: finishing write tid 2 to smithi09216568-20 2020-11-16T20:24:21.247 INFO:tasks.rados.rados.0.smithi092.stdout:77: finishing write tid 1 to smithi09216568-43 2020-11-16T20:24:21.247 INFO:tasks.rados.rados.0.smithi092.stdout:77: finishing write tid 2 to smithi09216568-43 2020-11-16T20:24:21.247 INFO:tasks.rados.rados.0.smithi092.stdout:77: finishing write tid 3 to smithi09216568-43 2020-11-16T20:24:21.247 INFO:tasks.rados.rados.0.smithi092.stdout:77: finishing write tid 4 to smithi09216568-43 2020-11-16T20:24:21.247 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 43 v 12 (ObjNum 56 snap 3 seq_num 56) dirty exists 2020-11-16T20:24:21.248 INFO:tasks.rados.rados.0.smithi092.stdout:77: left oid 43 (ObjNum 56 snap 3 seq_num 56) 2020-11-16T20:24:21.248 INFO:tasks.rados.rados.0.smithi092.stdout:78: finishing copy_from to smithi09216568-10 2020-11-16T20:24:21.248 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 10 v 15 (ObjNum 34 snap 0 seq_num 34) dirty exists 2020-11-16T20:24:21.248 INFO:tasks.rados.rados.0.smithi092.stdout:78: finishing copy_from racing read to smithi09216568-10 2020-11-16T20:24:21.248 INFO:tasks.rados.rados.0.smithi092.stdout:73: finishing write tid 3 to smithi09216568-20 2020-11-16T20:24:21.248 INFO:tasks.rados.rados.0.smithi092.stdout:73: finishing write tid 4 to smithi09216568-20 2020-11-16T20:24:21.248 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 20 v 10 (ObjNum 54 snap 3 seq_num 54) dirty exists 2020-11-16T20:24:21.249 INFO:tasks.rados.rados.0.smithi092.stdout:73: left oid 20 (ObjNum 54 snap 3 seq_num 54) 2020-11-16T20:24:21.249 INFO:tasks.rados.rados.0.smithi092.stdout:81: read oid 42 snap -1 2020-11-16T20:24:21.249 INFO:tasks.rados.rados.0.smithi092.stdout:81: expect (ObjNum 41 snap 0 seq_num 41) 2020-11-16T20:24:21.249 INFO:tasks.rados.rados.0.smithi092.stdout:73: done (4 left) 2020-11-16T20:24:21.249 INFO:tasks.rados.rados.0.smithi092.stdout:77: done (3 left) 2020-11-16T20:24:21.249 INFO:tasks.rados.rados.0.smithi092.stdout:78: done (2 left) 2020-11-16T20:24:21.249 INFO:tasks.rados.rados.0.smithi092.stdout:82: snap_create 2020-11-16T20:24:21.249 INFO:tasks.rados.rados.0.smithi092.stdout:81: expect (ObjNum 41 snap 0 seq_num 41) 2020-11-16T20:24:21.250 INFO:tasks.rados.rados.0.smithi092.stdout:80: expect (ObjNum 45 snap 0 seq_num 45) 2020-11-16T20:24:21.250 INFO:tasks.rados.rados.0.smithi092.stdout:80: done (2 left) 2020-11-16T20:24:21.250 INFO:tasks.rados.rados.0.smithi092.stdout:81: done (1 left) 2020-11-16T20:24:21.250 INFO:tasks.rados.rados.0.smithi092.stdout:82: done (0 left) 2020-11-16T20:24:21.250 INFO:tasks.rados.rados.0.smithi092.stdout:83: snap_remove snap 1 2020-11-16T20:24:21.250 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:21.021+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:21.251 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:19.584+0000 7f11d7a34700 -1 osd.7 pg_epoch: 35 pg[3.0s1( v 32'8 (0'0,32'8] local-lis/les=26/27 n=4 ec=23/23 lis/c=26/26 les/c/f=27/27/0 sis=26) [5,7,2]p7(1) r=1 lpr=26 crt=32'8 lcod 31'6 mlcod 31'6 active+clean+snaptrim trimq=[2~1]] removing snap head 2020-11-16T20:24:21.251 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-11-16T20:24:21.254 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:21.253+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:21.354 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:21.352+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:21.381 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:21.381 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:21.381 INFO:teuthology.orchestra.run.smithi013.stdout: "num_ops": 0 2020-11-16T20:24:21.381 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:21.392 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-11-16T20:24:21.407 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:21.431 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:21.454 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:21.477 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:21.476+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:21.478 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:21.517 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:21.552 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:21.552 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:21.553 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:21.555 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:21.552+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:21.563 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-11-16T20:24:21.579 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:21.588 INFO:tasks.rados.rados.0.smithi092.stdout:83: done (0 left) 2020-11-16T20:24:21.588 INFO:tasks.rados.rados.0.smithi092.stdout:84: snap_remove snap 2 2020-11-16T20:24:21.595 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2020-11-16T20:24:21.600 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:21.592+0000 7f935bee7700 -1 osd.4 pg_epoch: 37 pg[3.7s0( v 37'19 (0'0,37'19] local-lis/les=23/24 n=7 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) [4,6,1]p4(0) r=0 lpr=23 luod=34'17 crt=34'17 lcod 33'15 mlcod 33'15 active+clean+snaptrim trimq=[2~2]] removing snap head 2020-11-16T20:24:21.601 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2020-11-16T20:24:21.612 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2020-11-16T20:24:21.620 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:21.621 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T20:24:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T20:24:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T20:24:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-11-16T20:24:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-11-16T20:24:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-11-16T20:24:21.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-11-16T20:24:21.623 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-11-16T20:24:21.676 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:21.677+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:21.706 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:21.707 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:21.717 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-11-16T20:24:21.757 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:21.757+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:21.841 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:21.841 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:21.852 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-11-16T20:24:21.858 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:21.857+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:21.959 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:21.957+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:21.965 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:21.966 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:21.976 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-11-16T20:24:22.060 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:22.061+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:22.096 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:22.096 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:22.106 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-11-16T20:24:22.160 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:22.156+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:22.222 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:22.222 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:22.232 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-11-16T20:24:22.262 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:22.261+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:22.362 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:22.361+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:22.393 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:22.393 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:22.403 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-11-16T20:24:22.463 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:22.460+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:22.520 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:22.521 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:22.532 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-11-16T20:24:22.573 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:22.572+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:22.606 INFO:tasks.rados.rados.0.smithi092.stdout:84: done (0 left) 2020-11-16T20:24:22.607 INFO:tasks.rados.rados.0.smithi092.stdout:85: read oid 41 snap -1 2020-11-16T20:24:22.607 INFO:tasks.rados.rados.0.smithi092.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2020-11-16T20:24:22.607 INFO:tasks.rados.rados.0.smithi092.stdout:86: copy_from oid 7 from oid 45 current snap is 4 2020-11-16T20:24:22.607 INFO:tasks.rados.rados.0.smithi092.stdout:87: rollback oid 19 current snap is 4 2020-11-16T20:24:22.607 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 19 to 3 2020-11-16T20:24:22.607 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 37 current snap is 4 2020-11-16T20:24:22.608 INFO:tasks.rados.rados.0.smithi092.stdout:88: seq_num 57 ranges {0=557056,557056=401408} 2020-11-16T20:24:22.611 INFO:tasks.rados.rados.0.smithi092.stdout:88: writing smithi09216568-37 from 0 to 557056 tid 1 2020-11-16T20:24:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 1 (10000) 2020-11-16T20:24:22.614 INFO:tasks.rados.rados.0.smithi092.stdout:88: writing smithi09216568-37 from 557056 to 958464 tid 2 2020-11-16T20:24:22.615 INFO:tasks.rados.rados.0.smithi092.stdout:89: rollback oid 20 current snap is 4 2020-11-16T20:24:22.615 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 20 to 3 2020-11-16T20:24:22.615 INFO:tasks.rados.rados.0.smithi092.stdout:90: read oid 17 snap -1 2020-11-16T20:24:22.615 INFO:tasks.rados.rados.0.smithi092.stdout:90: expect (ObjNum 16 snap 0 seq_num 16) 2020-11-16T20:24:22.615 INFO:tasks.rados.rados.0.smithi092.stdout:91: read oid 15 snap -1 2020-11-16T20:24:22.616 INFO:tasks.rados.rados.0.smithi092.stdout:91: expect deleted 2020-11-16T20:24:22.616 INFO:tasks.rados.rados.0.smithi092.stdout:92: rollback oid 47 current snap is 4 2020-11-16T20:24:22.616 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 47 to 3 2020-11-16T20:24:22.616 INFO:tasks.rados.rados.0.smithi092.stdout:93: read oid 13 snap 3 2020-11-16T20:24:22.616 INFO:tasks.rados.rados.0.smithi092.stdout:93: expect (ObjNum 12 snap 0 seq_num 12) 2020-11-16T20:24:22.616 INFO:tasks.rados.rados.0.smithi092.stdout:94: snap_create 2020-11-16T20:24:22.619 INFO:tasks.rados.rados.0.smithi092.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2020-11-16T20:24:22.632 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2020-11-16T20:24:22.641 INFO:tasks.rados.rados.0.smithi092.stdout:87: finishing rollback tid 0 to smithi09216568-19 2020-11-16T20:24:22.641 INFO:tasks.rados.rados.0.smithi092.stdout:90: expect (ObjNum 16 snap 0 seq_num 16) 2020-11-16T20:24:22.649 INFO:tasks.rados.rados.0.smithi092.stdout:93: expect (ObjNum 12 snap 0 seq_num 12) 2020-11-16T20:24:22.654 INFO:tasks.rados.rados.0.smithi092.stdout:92: finishing rollback tid 0 to smithi09216568-47 2020-11-16T20:24:22.654 INFO:tasks.rados.rados.0.smithi092.stdout:89: finishing rollback tid 0 to smithi09216568-20 2020-11-16T20:24:22.655 INFO:tasks.rados.rados.0.smithi092.stdout:89: finishing rollback tid 1 to smithi09216568-20 2020-11-16T20:24:22.655 INFO:tasks.rados.rados.0.smithi092.stdout:92: finishing rollback tid 1 to smithi09216568-47 2020-11-16T20:24:22.655 INFO:tasks.rados.rados.0.smithi092.stdout:92: finishing rollback tid 2 to smithi09216568-47 2020-11-16T20:24:22.655 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 47 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2020-11-16T20:24:22.655 INFO:tasks.rados.rados.0.smithi092.stdout:88: finishing write tid 1 to smithi09216568-37 2020-11-16T20:24:22.655 INFO:tasks.rados.rados.0.smithi092.stdout:87: finishing rollback tid 1 to smithi09216568-19 2020-11-16T20:24:22.655 INFO:tasks.rados.rados.0.smithi092.stdout:87: finishing rollback tid 2 to smithi09216568-19 2020-11-16T20:24:22.656 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 16 (ObjNum 18 snap 0 seq_num 18) dirty exists 2020-11-16T20:24:22.656 INFO:tasks.rados.rados.0.smithi092.stdout:88: finishing write tid 2 to smithi09216568-37 2020-11-16T20:24:22.656 INFO:tasks.rados.rados.0.smithi092.stdout:86: finishing copy_from to smithi09216568-7 2020-11-16T20:24:22.656 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 7 v 10 (ObjNum 44 snap 0 seq_num 44) dirty exists 2020-11-16T20:24:22.656 INFO:tasks.rados.rados.0.smithi092.stdout:86: finishing copy_from racing read to smithi09216568-7 2020-11-16T20:24:22.664 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:22.665+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:22.711 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:22.711 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:22.722 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-11-16T20:24:22.769 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:22.768+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:22.835 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:22.836 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:22.847 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-11-16T20:24:22.887 INFO:tasks.rados.rados.0.smithi092.stdout:89: finishing rollback tid 2 to smithi09216568-20 2020-11-16T20:24:22.887 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 20 v 18 (ObjNum 54 snap 3 seq_num 54) dirty exists 2020-11-16T20:24:22.889 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:22.888+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:22.966 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:22.964+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:23.010 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:23.010 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:23.020 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T20:24:23.067 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:23.064+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:23.139 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:23.140 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:23.149 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T20:24:23.167 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:23.164+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:23.222 INFO:tasks.rados.rados.0.smithi092.stdout:88: finishing write tid 3 to smithi09216568-37 2020-11-16T20:24:23.227 INFO:tasks.rados.rados.0.smithi092.stdout:88: finishing write tid 4 to smithi09216568-37 2020-11-16T20:24:23.227 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 37 v 13 (ObjNum 57 snap 4 seq_num 57) dirty exists 2020-11-16T20:24:23.227 INFO:tasks.rados.rados.0.smithi092.stdout:88: left oid 37 (ObjNum 57 snap 4 seq_num 57) 2020-11-16T20:24:23.266 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:23.266 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:23.268 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:23.264+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:23.277 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T20:24:23.370 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:23.369+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:23.393 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:23.394 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:23.403 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T20:24:23.471 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:23.468+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:23.571 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:23.571 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:23.574 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:23.572+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:23.578 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T20:24:23.611 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:23.608+0000 7f7377480700 -1 osd.6 pg_epoch: 38 pg[3.1s0( v 38'28 (0'0,38'28] local-lis/les=23/24 n=7 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) [6,2,4]p6(0) r=0 lpr=23 luod=37'26 crt=37'26 lcod 37'24 mlcod 37'24 active+clean+snaptrim trimq=[4~1] ps=[2~2]] removing snap head 2020-11-16T20:24:23.620 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2020-11-16T20:24:23.622 INFO:tasks.rados.rados.0.smithi092.stdout:85: done (9 left) 2020-11-16T20:24:23.622 INFO:tasks.rados.rados.0.smithi092.stdout:86: done (8 left) 2020-11-16T20:24:23.623 INFO:tasks.rados.rados.0.smithi092.stdout:87: done (7 left) 2020-11-16T20:24:23.623 INFO:tasks.rados.rados.0.smithi092.stdout:88: done (6 left) 2020-11-16T20:24:23.623 INFO:tasks.rados.rados.0.smithi092.stdout:89: done (5 left) 2020-11-16T20:24:23.623 INFO:tasks.rados.rados.0.smithi092.stdout:90: done (4 left) 2020-11-16T20:24:23.623 INFO:tasks.rados.rados.0.smithi092.stdout:91: done (3 left) 2020-11-16T20:24:23.623 INFO:tasks.rados.rados.0.smithi092.stdout:92: done (2 left) 2020-11-16T20:24:23.623 INFO:tasks.rados.rados.0.smithi092.stdout:93: done (1 left) 2020-11-16T20:24:23.624 INFO:tasks.rados.rados.0.smithi092.stdout:94: done (0 left) 2020-11-16T20:24:23.624 INFO:tasks.rados.rados.0.smithi092.stdout:95: read oid 42 snap -1 2020-11-16T20:24:23.624 INFO:tasks.rados.rados.0.smithi092.stdout:95: expect (ObjNum 41 snap 0 seq_num 41) 2020-11-16T20:24:23.624 INFO:tasks.rados.rados.0.smithi092.stdout:96: setattr oid 5 current snap is 5 2020-11-16T20:24:23.625 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 38 current snap is 5 2020-11-16T20:24:23.625 INFO:tasks.rados.rados.0.smithi092.stdout:97: seq_num 58 ranges {622592=573440,1196032=401408} 2020-11-16T20:24:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2020-11-16T20:24:23.630 INFO:tasks.rados.rados.0.smithi092.stdout:97: writing smithi09216568-38 from 622592 to 1196032 tid 1 2020-11-16T20:24:23.633 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T20:24:23.634 INFO:tasks.rados.rados.0.smithi092.stdout:97: writing smithi09216568-38 from 1196032 to 1597440 tid 2 2020-11-16T20:24:23.639 INFO:tasks.rados.rados.0.smithi092.stdout:98: rollback oid 23 current snap is 5 2020-11-16T20:24:23.639 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 23 to 4 2020-11-16T20:24:23.639 INFO:tasks.rados.rados.0.smithi092.stdout:99: read oid 32 snap -1 2020-11-16T20:24:23.639 INFO:tasks.rados.rados.0.smithi092.stdout:99: expect (ObjNum 31 snap 0 seq_num 31) 2020-11-16T20:24:23.639 INFO:tasks.rados.rados.0.smithi092.stdout:100: setattr oid 49 current snap is 5 2020-11-16T20:24:23.640 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 11 current snap is 5 2020-11-16T20:24:23.640 INFO:tasks.rados.rados.0.smithi092.stdout:101: seq_num 59 ranges {90112=598016,688128=401408} 2020-11-16T20:24:23.641 INFO:tasks.rados.rados.0.smithi092.stdout:101: writing smithi09216568-11 from 90112 to 688128 tid 1 2020-11-16T20:24:23.644 INFO:tasks.rados.rados.0.smithi092.stdout:101: writing smithi09216568-11 from 688128 to 1089536 tid 2 2020-11-16T20:24:23.644 INFO:tasks.rados.rados.0.smithi092.stdout:102: rmattr oid 31 current snap is 5 2020-11-16T20:24:23.644 INFO:tasks.rados.rados.0.smithi092.stdout:98: finishing rollback tid 1 to smithi09216568-23 2020-11-16T20:24:23.644 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 23 v 0 (ObjNum 288 snap 0 seq_num 81) dirty dne 2020-11-16T20:24:23.644 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 27 (ObjNum 4 snap 0 seq_num 4) dirty exists 2020-11-16T20:24:23.645 INFO:tasks.rados.rados.0.smithi092.stdout:95: expect (ObjNum 41 snap 0 seq_num 41) 2020-11-16T20:24:23.665 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2020-11-16T20:24:23.671 INFO:tasks.rados.rados.0.smithi092.stdout:95: done (7 left) 2020-11-16T20:24:23.671 INFO:tasks.rados.rados.0.smithi092.stdout:96: done (6 left) 2020-11-16T20:24:23.671 INFO:tasks.rados.rados.0.smithi092.stdout:98: done (5 left) 2020-11-16T20:24:23.671 INFO:tasks.rados.rados.0.smithi092.stdout:103: rollback oid 8 current snap is 5 2020-11-16T20:24:23.671 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 8 to 4 2020-11-16T20:24:23.671 INFO:tasks.rados.rados.0.smithi092.stdout:99: expect (ObjNum 31 snap 0 seq_num 31) 2020-11-16T20:24:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 49 v 13 (ObjNum 48 snap 0 seq_num 48) dirty exists 2020-11-16T20:24:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 31 v 10 (ObjNum 30 snap 0 seq_num 30) dirty exists 2020-11-16T20:24:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:101: finishing write tid 1 to smithi09216568-11 2020-11-16T20:24:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:97: finishing write tid 1 to smithi09216568-38 2020-11-16T20:24:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:97: finishing write tid 2 to smithi09216568-38 2020-11-16T20:24:23.694 INFO:tasks.rados.rados.0.smithi092.stdout:101: finishing write tid 2 to smithi09216568-11 2020-11-16T20:24:23.695 INFO:tasks.rados.rados.0.smithi092.stdout:101: finishing write tid 3 to smithi09216568-11 2020-11-16T20:24:23.695 INFO:tasks.rados.rados.0.smithi092.stdout:99: done (5 left) 2020-11-16T20:24:23.695 INFO:tasks.rados.rados.0.smithi092.stdout:100: done (4 left) 2020-11-16T20:24:23.695 INFO:tasks.rados.rados.0.smithi092.stdout:102: done (3 left) 2020-11-16T20:24:23.695 INFO:tasks.rados.rados.0.smithi092.stdout:104: snap_remove snap 3 2020-11-16T20:24:23.696 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:23.693+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:23.758 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:23.758 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:23.773 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T20:24:23.775 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:23.772+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:23.815 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:23.845 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:23.867 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:23.877 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:23.877+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:23.891 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:23.919 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:23.943 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:23.943 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:23.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:23.943 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:23.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:1 1.0 1:08c0073d:::INTEL_SSDPED1D480GA_PHMB7476008K480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2020-11-16T20:24:23.944 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:23.944 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.267808043000002, 2020-11-16T20:24:23.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1045939339999999, 2020-11-16T20:24:23.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:23.944 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:23.944 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:23.944 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:23.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:23.945 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1 2020-11-16T20:24:23.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.945 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:23.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:23.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:23.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:23.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:23.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675868+0000", 2020-11-16T20:24:23.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5326000000000004e-05 2020-11-16T20:24:23.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:23.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675933+0000", 2020-11-16T20:24:23.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2425999999999999e-05 2020-11-16T20:24:23.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:23.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675946+0000", 2020-11-16T20:24:23.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3563000000000001e-05 2020-11-16T20:24:23.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:23.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675969+0000", 2020-11-16T20:24:23.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4087000000000001e-05 2020-11-16T20:24:23.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.675983+0000", 2020-11-16T20:24:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011954499999999999 2020-11-16T20:24:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:23.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.676103+0000", 2020-11-16T20:24:23.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.76e-05 2020-11-16T20:24:23.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2020-11-16T20:24:23.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:38.676130+0000", 2020-11-16T20:24:23.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1039924210000001 2020-11-16T20:24:23.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:23.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.780123+0000", 2020-11-16T20:24:23.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033896600000000001 2020-11-16T20:24:23.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.780462+0000", 2020-11-16T20:24:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1045939339999999 2020-11-16T20:24:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4207.0:4 1.0 1:97e83c24:::INTEL_SSDPEDMD400G4_PHFT620400QM400BGN:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e14)", 2020-11-16T20:24:23.953 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:23.953 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 44.028619792000001, 2020-11-16T20:24:23.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045371300000000003, 2020-11-16T20:24:23.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:23.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:23.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:23.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4207", 2020-11-16T20:24:23.954 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/2647985223", 2020-11-16T20:24:23.954 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 4 2020-11-16T20:24:23.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.954 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:23.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:23.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:23.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:23.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:23.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915056+0000", 2020-11-16T20:24:23.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3885000000000002e-05 2020-11-16T20:24:23.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:23.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915130+0000", 2020-11-16T20:24:23.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.29e-05 2020-11-16T20:24:23.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:23.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915143+0000", 2020-11-16T20:24:23.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9257000000000001e-05 2020-11-16T20:24:23.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:23.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915172+0000", 2020-11-16T20:24:23.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9451999999999999e-05 2020-11-16T20:24:23.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:23.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915192+0000", 2020-11-16T20:24:23.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011178700000000001 2020-11-16T20:24:23.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:23.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915303+0000", 2020-11-16T20:24:23.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031313000000000001 2020-11-16T20:24:23.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:23.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915617+0000", 2020-11-16T20:24:23.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029708700000000002 2020-11-16T20:24:23.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2020-11-16T20:24:23.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.915914+0000", 2020-11-16T20:24:23.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015209430000000001 2020-11-16T20:24:23.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2020-11-16T20:24:23.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.917435+0000", 2020-11-16T20:24:23.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018238799999999999 2020-11-16T20:24:23.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2020-11-16T20:24:23.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919258+0000", 2020-11-16T20:24:23.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000208672 2020-11-16T20:24:23.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:23.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919467+0000", 2020-11-16T20:24:23.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012613699999999999 2020-11-16T20:24:23.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:23.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:23:39.919593+0000", 2020-11-16T20:24:23.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045371300000000003 2020-11-16T20:24:23.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.963 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:23.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.963 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 31/23 ECSubWrite(tid=1, reqid=client.4444.0:22, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:23.963 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.534302+0000", 2020-11-16T20:24:23.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4093738810000005, 2020-11-16T20:24:23.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004870625, 2020-11-16T20:24:23.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:23.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:23.964 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:23.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:23.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534302+0000", 2020-11-16T20:24:23.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:23.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:23.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534302+0000", 2020-11-16T20:24:23.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000195269 2020-11-16T20:24:23.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:23.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534497+0000", 2020-11-16T20:24:23.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5259e-05 2020-11-16T20:24:23.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:23.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534523+0000", 2020-11-16T20:24:23.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9455000000000002e-05 2020-11-16T20:24:23.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:23.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534622+0000", 2020-11-16T20:24:23.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0397000000000001e-05 2020-11-16T20:24:23.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:23.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534632+0000", 2020-11-16T20:24:23.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4876000000000006e-05 2020-11-16T20:24:23.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:23.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534707+0000", 2020-11-16T20:24:23.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3736e-05 2020-11-16T20:24:23.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:23.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.534721+0000", 2020-11-16T20:24:23.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043959799999999999 2020-11-16T20:24:23.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:23.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539117+0000", 2020-11-16T20:24:23.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5652999999999997e-05 2020-11-16T20:24:23.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:23.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539173+0000", 2020-11-16T20:24:23.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004870625 2020-11-16T20:24:23.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.971 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:23.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 31/23 ECSubWrite(tid=3, reqid=client.4444.0:37, at_version=31'2, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:23.972 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.544872+0000", 2020-11-16T20:24:23.972 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3988039210000007, 2020-11-16T20:24:23.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065869739999999998, 2020-11-16T20:24:23.972 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:23.972 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:23.972 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:23.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:23.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.544872+0000", 2020-11-16T20:24:23.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:23.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:23.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.544872+0000", 2020-11-16T20:24:23.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012931100000000001 2020-11-16T20:24:23.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:23.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545001+0000", 2020-11-16T20:24:23.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5837e-05 2020-11-16T20:24:23.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545027+0000", 2020-11-16T20:24:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1269999999999999e-05 2020-11-16T20:24:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545108+0000", 2020-11-16T20:24:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7259999999999997e-06 2020-11-16T20:24:23.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:23.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545118+0000", 2020-11-16T20:24:23.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9775000000000003e-05 2020-11-16T20:24:23.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:23.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545178+0000", 2020-11-16T20:24:23.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2670999999999999e-05 2020-11-16T20:24:23.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:23.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.545191+0000", 2020-11-16T20:24:23.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062062649999999999 2020-11-16T20:24:23.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:23.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551397+0000", 2020-11-16T20:24:23.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2119000000000006e-05 2020-11-16T20:24:23.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:23.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551459+0000", 2020-11-16T20:24:23.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065869739999999998 2020-11-16T20:24:23.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.979 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:23.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.980 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=3, reqid=client.4444.0:43, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:23.980 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.553831+0000", 2020-11-16T20:24:23.980 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3898450950000001, 2020-11-16T20:24:23.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052721310000000002, 2020-11-16T20:24:23.980 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:23.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:23.980 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:23.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:23.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.553831+0000", 2020-11-16T20:24:23.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:23.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:23.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.553831+0000", 2020-11-16T20:24:23.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087011200000000003 2020-11-16T20:24:23.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:23.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554701+0000", 2020-11-16T20:24:23.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2575e-05 2020-11-16T20:24:23.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:23.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554724+0000", 2020-11-16T20:24:23.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8412999999999996e-05 2020-11-16T20:24:23.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:23.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554802+0000", 2020-11-16T20:24:23.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2260000000000005e-06 2020-11-16T20:24:23.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:23.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554810+0000", 2020-11-16T20:24:23.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2891000000000004e-05 2020-11-16T20:24:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554883+0000", 2020-11-16T20:24:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1015999999999999e-05 2020-11-16T20:24:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:23.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.554894+0000", 2020-11-16T20:24:23.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004148746 2020-11-16T20:24:23.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:23.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.559043+0000", 2020-11-16T20:24:23.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0152000000000003e-05 2020-11-16T20:24:23.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:23.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.559103+0000", 2020-11-16T20:24:23.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052721310000000002 2020-11-16T20:24:23.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.987 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:23.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.988 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 31/23 ECSubWrite(tid=7, reqid=client.4444.0:38, at_version=31'4, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:23.988 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.557239+0000", 2020-11-16T20:24:23.988 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3864367620000007, 2020-11-16T20:24:23.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043101739999999999, 2020-11-16T20:24:23.988 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557239+0000", 2020-11-16T20:24:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:23.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557239+0000", 2020-11-16T20:24:23.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3023999999999998e-05 2020-11-16T20:24:23.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:23.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557302+0000", 2020-11-16T20:24:23.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4273e-05 2020-11-16T20:24:23.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:23.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557327+0000", 2020-11-16T20:24:23.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2374999999999997e-05 2020-11-16T20:24:23.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:23.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557409+0000", 2020-11-16T20:24:23.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3429999999999996e-06 2020-11-16T20:24:23.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:23.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557417+0000", 2020-11-16T20:24:23.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031612989999999998 2020-11-16T20:24:23.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:23.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560579+0000", 2020-11-16T20:24:23.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9585999999999997e-05 2020-11-16T20:24:23.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:23.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560618+0000", 2020-11-16T20:24:23.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086567000000000002 2020-11-16T20:24:23.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:23.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.561484+0000", 2020-11-16T20:24:23.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5604000000000004e-05 2020-11-16T20:24:23.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:23.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.561549+0000", 2020-11-16T20:24:23.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043101739999999999 2020-11-16T20:24:23.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.996 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:23.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:23.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.996 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=15, reqid=client.4444.0:61, at_version=31'3, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:23.996 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.581005+0000", 2020-11-16T20:24:23.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3626713049999992, 2020-11-16T20:24:23.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059672299999999996, 2020-11-16T20:24:23.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:23.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:23.997 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:23.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:23.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581005+0000", 2020-11-16T20:24:23.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581005+0000", 2020-11-16T20:24:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026132899999999998 2020-11-16T20:24:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:23.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581266+0000", 2020-11-16T20:24:23.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4717000000000001e-05 2020-11-16T20:24:23.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:23.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:23.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:23.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581291+0000", 2020-11-16T20:24:23.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6506000000000002e-05 2020-11-16T20:24:24.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581367+0000", 2020-11-16T20:24:24.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2019999999999993e-06 2020-11-16T20:24:24.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581376+0000", 2020-11-16T20:24:24.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1564999999999998e-05 2020-11-16T20:24:24.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581438+0000", 2020-11-16T20:24:24.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4117999999999999e-05 2020-11-16T20:24:24.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.581452+0000", 2020-11-16T20:24:24.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054574990000000002 2020-11-16T20:24:24.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.586910+0000", 2020-11-16T20:24:24.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2293999999999997e-05 2020-11-16T20:24:24.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.586972+0000", 2020-11-16T20:24:24.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059672299999999996 2020-11-16T20:24:24.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.004 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.004 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=18, reqid=client.4444.0:70, at_version=31'4, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:24.004 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.588330+0000", 2020-11-16T20:24:24.004 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.355346462, 2020-11-16T20:24:24.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012654103, 2020-11-16T20:24:24.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.005 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588330+0000", 2020-11-16T20:24:24.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588330+0000", 2020-11-16T20:24:24.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001257293 2020-11-16T20:24:24.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589587+0000", 2020-11-16T20:24:24.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7559000000000002e-05 2020-11-16T20:24:24.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589614+0000", 2020-11-16T20:24:24.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6135000000000002e-05 2020-11-16T20:24:24.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589690+0000", 2020-11-16T20:24:24.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0029e-05 2020-11-16T20:24:24.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589701+0000", 2020-11-16T20:24:24.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2075000000000003e-05 2020-11-16T20:24:24.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589783+0000", 2020-11-16T20:24:24.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8433000000000001e-05 2020-11-16T20:24:24.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589801+0000", 2020-11-16T20:24:24.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011126036000000001 2020-11-16T20:24:24.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600927+0000", 2020-11-16T20:24:24.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6542999999999997e-05 2020-11-16T20:24:24.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600984+0000", 2020-11-16T20:24:24.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012654103 2020-11-16T20:24:24.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.012 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.012 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s2 31/23 ECSubWrite(tid=21, reqid=client.4444.0:79, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:24.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.600262+0000", 2020-11-16T20:24:24.013 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3434140190000008, 2020-11-16T20:24:24.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013056373, 2020-11-16T20:24:24.013 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.013 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.013 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600262+0000", 2020-11-16T20:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600262+0000", 2020-11-16T20:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046778800000000001 2020-11-16T20:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600730+0000", 2020-11-16T20:24:24.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4491e-05 2020-11-16T20:24:24.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600754+0000", 2020-11-16T20:24:24.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2707999999999995e-05 2020-11-16T20:24:24.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600837+0000", 2020-11-16T20:24:24.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1890000000000001e-06 2020-11-16T20:24:24.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600844+0000", 2020-11-16T20:24:24.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7360999999999994e-05 2020-11-16T20:24:24.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600911+0000", 2020-11-16T20:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.113e-05 2020-11-16T20:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.600923+0000", 2020-11-16T20:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012339085 2020-11-16T20:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613262+0000", 2020-11-16T20:24:24.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6620999999999999e-05 2020-11-16T20:24:24.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613318+0000", 2020-11-16T20:24:24.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013056373 2020-11-16T20:24:24.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=24, reqid=client.4444.0:71, at_version=31'6, trim_to=0'0, roll_forward_to=31'4))", 2020-11-16T20:24:24.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.602830+0000", 2020-11-16T20:24:24.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3408462290000003, 2020-11-16T20:24:24.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046210660000000001, 2020-11-16T20:24:24.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602830+0000", 2020-11-16T20:24:24.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602830+0000", 2020-11-16T20:24:24.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5114999999999997e-05 2020-11-16T20:24:24.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602885+0000", 2020-11-16T20:24:24.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2595999999999998e-05 2020-11-16T20:24:24.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602907+0000", 2020-11-16T20:24:24.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4755999999999999e-05 2020-11-16T20:24:24.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603002+0000", 2020-11-16T20:24:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0198e-05 2020-11-16T20:24:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603012+0000", 2020-11-16T20:24:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9153000000000001e-05 2020-11-16T20:24:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603072+0000", 2020-11-16T20:24:24.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3925e-05 2020-11-16T20:24:24.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603085+0000", 2020-11-16T20:24:24.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043104240000000002 2020-11-16T20:24:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.607396+0000", 2020-11-16T20:24:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4898999999999997e-05 2020-11-16T20:24:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.607451+0000", 2020-11-16T20:24:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046210660000000001 2020-11-16T20:24:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.fs1 31/23 ECSubWrite(tid=11, reqid=client.4444.0:94, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.614607+0000", 2020-11-16T20:24:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3290693279999992, 2020-11-16T20:24:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007262976, 2020-11-16T20:24:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614607+0000", 2020-11-16T20:24:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614607+0000", 2020-11-16T20:24:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014703399999999999 2020-11-16T20:24:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614754+0000", 2020-11-16T20:24:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0153000000000001e-05 2020-11-16T20:24:24.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614784+0000", 2020-11-16T20:24:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9525999999999997e-05 2020-11-16T20:24:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614863+0000", 2020-11-16T20:24:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3206e-05 2020-11-16T20:24:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614877+0000", 2020-11-16T20:24:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4196000000000001e-05 2020-11-16T20:24:24.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614931+0000", 2020-11-16T20:24:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1345000000000001e-05 2020-11-16T20:24:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614942+0000", 2020-11-16T20:24:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068665319999999998 2020-11-16T20:24:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.621809+0000", 2020-11-16T20:24:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0983999999999998e-05 2020-11-16T20:24:24.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.621870+0000", 2020-11-16T20:24:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007262976 2020-11-16T20:24:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s2 31/23 ECSubWrite(tid=28, reqid=client.4444.0:91, at_version=31'2, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.614836+0000", 2020-11-16T20:24:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3288397449999998, 2020-11-16T20:24:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017114507000000001, 2020-11-16T20:24:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614836+0000", 2020-11-16T20:24:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614836+0000", 2020-11-16T20:24:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043176289999999999 2020-11-16T20:24:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619154+0000", 2020-11-16T20:24:24.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5164000000000002e-05 2020-11-16T20:24:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619179+0000", 2020-11-16T20:24:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014133 2020-11-16T20:24:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619320+0000", 2020-11-16T20:24:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3549999999999994e-06 2020-11-16T20:24:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619330+0000", 2020-11-16T20:24:24.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5703000000000004e-05 2020-11-16T20:24:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619395+0000", 2020-11-16T20:24:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3317e-05 2020-11-16T20:24:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.619409+0000", 2020-11-16T20:24:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012482895000000001 2020-11-16T20:24:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631892+0000", 2020-11-16T20:24:24.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9113999999999997e-05 2020-11-16T20:24:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631951+0000", 2020-11-16T20:24:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017114507000000001 2020-11-16T20:24:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s2 31/23 ECSubWrite(tid=30, reqid=client.4444.0:80, at_version=31'3, trim_to=0'0, roll_forward_to=31'1))", 2020-11-16T20:24:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.620547+0000", 2020-11-16T20:24:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3231291059999997, 2020-11-16T20:24:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011500491, 2020-11-16T20:24:24.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620547+0000", 2020-11-16T20:24:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620547+0000", 2020-11-16T20:24:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3017999999999998e-05 2020-11-16T20:24:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620620+0000", 2020-11-16T20:24:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2254e-05 2020-11-16T20:24:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620652+0000", 2020-11-16T20:24:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4431000000000006e-05 2020-11-16T20:24:24.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620727+0000", 2020-11-16T20:24:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.554e-06 2020-11-16T20:24:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.620734+0000", 2020-11-16T20:24:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039200349999999997 2020-11-16T20:24:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.624654+0000", 2020-11-16T20:24:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3179999999999999e-05 2020-11-16T20:24:24.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.624667+0000", 2020-11-16T20:24:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073281939999999997 2020-11-16T20:24:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631996+0000", 2020-11-16T20:24:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1824999999999999e-05 2020-11-16T20:24:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.632047+0000", 2020-11-16T20:24:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011500491 2020-11-16T20:24:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.052 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.as1 31/23 ECSubWrite(tid=17, reqid=client.4444.0:106, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.635521+0000", 2020-11-16T20:24:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.308155417, 2020-11-16T20:24:24.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089614870000000006, 2020-11-16T20:24:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635521+0000", 2020-11-16T20:24:24.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635521+0000", 2020-11-16T20:24:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000163449 2020-11-16T20:24:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635684+0000", 2020-11-16T20:24:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2008999999999998e-05 2020-11-16T20:24:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635716+0000", 2020-11-16T20:24:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111994 2020-11-16T20:24:24.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635828+0000", 2020-11-16T20:24:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6809999999999998e-06 2020-11-16T20:24:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635836+0000", 2020-11-16T20:24:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1889000000000003e-05 2020-11-16T20:24:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635898+0000", 2020-11-16T20:24:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2262000000000001e-05 2020-11-16T20:24:24.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635910+0000", 2020-11-16T20:24:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085164839999999995 2020-11-16T20:24:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.644426+0000", 2020-11-16T20:24:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5719e-05 2020-11-16T20:24:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.644482+0000", 2020-11-16T20:24:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089614870000000006 2020-11-16T20:24:24.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=44, reqid=client.4444.0:142, at_version=31'7, trim_to=0'0, roll_forward_to=31'6))", 2020-11-16T20:24:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710694+0000", 2020-11-16T20:24:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.2329819979999996, 2020-11-16T20:24:24.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010166807, 2020-11-16T20:24:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710694+0000", 2020-11-16T20:24:24.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710694+0000", 2020-11-16T20:24:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085290500000000005 2020-11-16T20:24:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711547+0000", 2020-11-16T20:24:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2960999999999998e-05 2020-11-16T20:24:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711580+0000", 2020-11-16T20:24:24.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011092299999999999 2020-11-16T20:24:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711691+0000", 2020-11-16T20:24:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2792999999999999e-05 2020-11-16T20:24:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711704+0000", 2020-11-16T20:24:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3614000000000005e-05 2020-11-16T20:24:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711767+0000", 2020-11-16T20:24:24.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3365e-05 2020-11-16T20:24:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711781+0000", 2020-11-16T20:24:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090371310000000003 2020-11-16T20:24:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.720818+0000", 2020-11-16T20:24:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3115000000000003e-05 2020-11-16T20:24:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.720861+0000", 2020-11-16T20:24:24.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010166807 2020-11-16T20:24:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=48, reqid=client.4444.0:149, at_version=31'8, trim_to=0'0, roll_forward_to=31'6))", 2020-11-16T20:24:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.721081+0000", 2020-11-16T20:24:24.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.2225944729999991, 2020-11-16T20:24:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070510520000000004, 2020-11-16T20:24:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.721081+0000", 2020-11-16T20:24:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.721081+0000", 2020-11-16T20:24:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019835799999999999 2020-11-16T20:24:24.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.721280+0000", 2020-11-16T20:24:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9286e-05 2020-11-16T20:24:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.721309+0000", 2020-11-16T20:24:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017232 2020-11-16T20:24:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.721481+0000", 2020-11-16T20:24:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0268999999999999e-05 2020-11-16T20:24:24.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.721492+0000", 2020-11-16T20:24:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1501000000000006e-05 2020-11-16T20:24:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.721553+0000", 2020-11-16T20:24:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3643e-05 2020-11-16T20:24:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.721567+0000", 2020-11-16T20:24:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065111850000000001 2020-11-16T20:24:24.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.728078+0000", 2020-11-16T20:24:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4490000000000002e-05 2020-11-16T20:24:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.728133+0000", 2020-11-16T20:24:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070510520000000004 2020-11-16T20:24:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 31/23 ECSubWrite(tid=50, reqid=client.4444.0:150, at_version=31'10, trim_to=0'0, roll_forward_to=31'8))", 2020-11-16T20:24:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.734644+0000", 2020-11-16T20:24:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.2090315569999994, 2020-11-16T20:24:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045981299999999998, 2020-11-16T20:24:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.734644+0000", 2020-11-16T20:24:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.734644+0000", 2020-11-16T20:24:24.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018572900000000001 2020-11-16T20:24:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.734830+0000", 2020-11-16T20:24:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4209999999999999e-05 2020-11-16T20:24:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.734864+0000", 2020-11-16T20:24:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118322 2020-11-16T20:24:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.734983+0000", 2020-11-16T20:24:24.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4171999999999999e-05 2020-11-16T20:24:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.734997+0000", 2020-11-16T20:24:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.965e-05 2020-11-16T20:24:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735056+0000", 2020-11-16T20:24:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1749999999999999e-06 2020-11-16T20:24:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735064+0000", 2020-11-16T20:24:24.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041388029999999999 2020-11-16T20:24:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.739202+0000", 2020-11-16T20:24:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0068999999999999e-05 2020-11-16T20:24:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.739243+0000", 2020-11-16T20:24:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045981299999999998 2020-11-16T20:24:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.7s2 33/23 ECSubWrite(tid=64, reqid=client.4444.0:182, at_version=33'15, trim_to=0'0, roll_forward_to=33'13))", 2020-11-16T20:24:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:17.593267+0000", 2020-11-16T20:24:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.3504094530000001, 2020-11-16T20:24:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070243399999999996, 2020-11-16T20:24:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.086 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.593267+0000", 2020-11-16T20:24:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.593267+0000", 2020-11-16T20:24:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015502300000000001 2020-11-16T20:24:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.593422+0000", 2020-11-16T20:24:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7271999999999999e-05 2020-11-16T20:24:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.593449+0000", 2020-11-16T20:24:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5943000000000001e-05 2020-11-16T20:24:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.593525+0000", 2020-11-16T20:24:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9519999999999997e-06 2020-11-16T20:24:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.593532+0000", 2020-11-16T20:24:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7395000000000001e-05 2020-11-16T20:24:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.593599+0000", 2020-11-16T20:24:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1590000000000001e-05 2020-11-16T20:24:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.593611+0000", 2020-11-16T20:24:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065821680000000002 2020-11-16T20:24:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.600193+0000", 2020-11-16T20:24:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7997000000000004e-05 2020-11-16T20:24:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:17.600291+0000", 2020-11-16T20:24:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070243399999999996 2020-11-16T20:24:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/23 ECSubWrite(tid=44, reqid=osd.3.0:44, at_version=35'10, trim_to=0'0, roll_forward_to=34'8))", 2020-11-16T20:24:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:19.579779+0000", 2020-11-16T20:24:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3638973070000002, 2020-11-16T20:24:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050299129999999996, 2020-11-16T20:24:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.579779+0000", 2020-11-16T20:24:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.579779+0000", 2020-11-16T20:24:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2848e-05 2020-11-16T20:24:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.579832+0000", 2020-11-16T20:24:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2529e-05 2020-11-16T20:24:24.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.579864+0000", 2020-11-16T20:24:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9371999999999997e-05 2020-11-16T20:24:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.579963+0000", 2020-11-16T20:24:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0010000000000001e-05 2020-11-16T20:24:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.579973+0000", 2020-11-16T20:24:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092797699999999997 2020-11-16T20:24:24.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.580901+0000", 2020-11-16T20:24:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.294e-05 2020-11-16T20:24:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.580914+0000", 2020-11-16T20:24:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003834437 2020-11-16T20:24:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.584749+0000", 2020-11-16T20:24:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9799999999999997e-05 2020-11-16T20:24:24.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.584809+0000", 2020-11-16T20:24:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050299129999999996 2020-11-16T20:24:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 35/23 ECSubWrite(tid=46, reqid=osd.3.0:44, at_version=0'0, trim_to=0'0, roll_forward_to=35'10))", 2020-11-16T20:24:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:19.585768+0000", 2020-11-16T20:24:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3579084290000001, 2020-11-16T20:24:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048970439999999997, 2020-11-16T20:24:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.102 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.585768+0000", 2020-11-16T20:24:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.585768+0000", 2020-11-16T20:24:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9317000000000001e-05 2020-11-16T20:24:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.585797+0000", 2020-11-16T20:24:24.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1874e-05 2020-11-16T20:24:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.585809+0000", 2020-11-16T20:24:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0279999999999999e-05 2020-11-16T20:24:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.585869+0000", 2020-11-16T20:24:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2028000000000001e-05 2020-11-16T20:24:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.585881+0000", 2020-11-16T20:24:24.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7639000000000007e-05 2020-11-16T20:24:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.585959+0000", 2020-11-16T20:24:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3520000000000007e-06 2020-11-16T20:24:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.585967+0000", 2020-11-16T20:24:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00463737 2020-11-16T20:24:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.590604+0000", 2020-11-16T20:24:24.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0183999999999998e-05 2020-11-16T20:24:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.590665+0000", 2020-11-16T20:24:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0048970439999999997 2020-11-16T20:24:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.110 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.110 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:24.111 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:24.112 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:24.112 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:24.112 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T20:24:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T20:24:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T20:24:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T20:24:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-11-16T20:24:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-11-16T20:24:24.113 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-11-16T20:24:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-11-16T20:24:24.114 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-11-16T20:24:24.114 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:24.108+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:24.212 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:24.208+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:24.276 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:24.276 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:24.276 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:24.276 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:24.286 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T20:24:24.312 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:24.312+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:24.399 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:24.399 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:24.399 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:24.400 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:24.400 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:24.408 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T20:24:24.452 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:24.453+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:24.513 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:24.512+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:24.522 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:24.522 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:24.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:24.523 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:24.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.523 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 30/26 ECSubWrite(tid=0, reqid=client.4444.0:1, at_version=30'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:24.523 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:24.523 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9888585009999993, 2020-11-16T20:24:24.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110371539, 2020-11-16T20:24:24.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.524 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:24.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:24.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1965999999999994e-05 2020-11-16T20:24:24.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532961+0000", 2020-11-16T20:24:24.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4882000000000002e-05 2020-11-16T20:24:24.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532996+0000", 2020-11-16T20:24:24.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.107939761 2020-11-16T20:24:24.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640936+0000", 2020-11-16T20:24:24.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3749999999999997e-06 2020-11-16T20:24:24.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640943+0000", 2020-11-16T20:24:24.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000229192 2020-11-16T20:24:24.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641172+0000", 2020-11-16T20:24:24.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3837e-05 2020-11-16T20:24:24.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641186+0000", 2020-11-16T20:24:24.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002004053 2020-11-16T20:24:24.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643190+0000", 2020-11-16T20:24:24.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0473000000000002e-05 2020-11-16T20:24:24.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643251+0000", 2020-11-16T20:24:24.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110371539 2020-11-16T20:24:24.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.531 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.531 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:54 3.5s0 3:aad7141b:::smithi09216568-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:24.531 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:24.531 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9476637879999998, 2020-11-16T20:24:24.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027636214999999999, 2020-11-16T20:24:24.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:24.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.532 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 54 2020-11-16T20:24:24.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:24.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:24.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7353e-05 2020-11-16T20:24:24.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574111+0000", 2020-11-16T20:24:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1509999999999998e-06 2020-11-16T20:24:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574118+0000", 2020-11-16T20:24:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2278e-05 2020-11-16T20:24:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574131+0000", 2020-11-16T20:24:24.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3789999999999997e-06 2020-11-16T20:24:24.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574136+0000", 2020-11-16T20:24:24.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054445750000000001 2020-11-16T20:24:24.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.579581+0000", 2020-11-16T20:24:24.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117794 2020-11-16T20:24:24.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.579698+0000", 2020-11-16T20:24:24.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056929220000000004 2020-11-16T20:24:24.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.585391+0000", 2020-11-16T20:24:24.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106148 2020-11-16T20:24:24.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.585498+0000", 2020-11-16T20:24:24.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097071099999999997 2020-11-16T20:24:24.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.595205+0000", 2020-11-16T20:24:24.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013155099999999999 2020-11-16T20:24:24.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.595336+0000", 2020-11-16T20:24:24.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063739540000000003 2020-11-16T20:24:24.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601710+0000", 2020-11-16T20:24:24.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027636214999999999 2020-11-16T20:24:24.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.542 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.542 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:59 3.es0 3:746783df:::smithi09216568-20:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:24.543 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:24.543 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9471966569999992, 2020-11-16T20:24:24.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026665233, 2020-11-16T20:24:24.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:24.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.544 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.544 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59 2020-11-16T20:24:24.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.544 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:24.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5383999999999997e-05 2020-11-16T20:24:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574596+0000", 2020-11-16T20:24:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999998e-06 2020-11-16T20:24:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574604+0000", 2020-11-16T20:24:24.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3088000000000001e-05 2020-11-16T20:24:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574617+0000", 2020-11-16T20:24:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1129999999999997e-06 2020-11-16T20:24:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574623+0000", 2020-11-16T20:24:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007628709 2020-11-16T20:24:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582252+0000", 2020-11-16T20:24:24.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011149099999999999 2020-11-16T20:24:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582363+0000", 2020-11-16T20:24:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069613909999999999 2020-11-16T20:24:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589325+0000", 2020-11-16T20:24:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0105000000000001e-05 2020-11-16T20:24:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589385+0000", 2020-11-16T20:24:24.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052031099999999997 2020-11-16T20:24:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589905+0000", 2020-11-16T20:24:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072848720000000004 2020-11-16T20:24:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.597190+0000", 2020-11-16T20:24:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00390829 2020-11-16T20:24:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601098+0000", 2020-11-16T20:24:24.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108199 2020-11-16T20:24:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601206+0000", 2020-11-16T20:24:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026665233 2020-11-16T20:24:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.553 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:60 3.es0 3:746783df:::smithi09216568-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9470647159999999, 2020-11-16T20:24:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029540154999999998, 2020-11-16T20:24:24.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 60 2020-11-16T20:24:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.555 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9110000000000003e-05 2020-11-16T20:24:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574712+0000", 2020-11-16T20:24:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8739999999999997e-06 2020-11-16T20:24:24.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574719+0000", 2020-11-16T20:24:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2554e-05 2020-11-16T20:24:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574732+0000", 2020-11-16T20:24:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-06 2020-11-16T20:24:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574736+0000", 2020-11-16T20:24:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077321949999999999 2020-11-16T20:24:24.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582469+0000", 2020-11-16T20:24:24.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010602 2020-11-16T20:24:24.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582575+0000", 2020-11-16T20:24:24.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078916549999999992 2020-11-16T20:24:24.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.590466+0000", 2020-11-16T20:24:24.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107569 2020-11-16T20:24:24.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.590574+0000", 2020-11-16T20:24:24.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010689094999999999 2020-11-16T20:24:24.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601263+0000", 2020-11-16T20:24:24.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7796999999999999e-05 2020-11-16T20:24:24.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601361+0000", 2020-11-16T20:24:24.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028524959999999999 2020-11-16T20:24:24.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604213+0000", 2020-11-16T20:24:24.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029540154999999998 2020-11-16T20:24:24.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.564 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:66 3.es0 3:7fa2fe22:::smithi09216568-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:24.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.578915+0000", 2020-11-16T20:24:24.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9428225680000004, 2020-11-16T20:24:24.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025562700000000001, 2020-11-16T20:24:24.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:24.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 66 2020-11-16T20:24:24.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578915+0000", 2020-11-16T20:24:24.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578915+0000", 2020-11-16T20:24:24.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4132000000000003e-05 2020-11-16T20:24:24.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578969+0000", 2020-11-16T20:24:24.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0862e-05 2020-11-16T20:24:24.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578980+0000", 2020-11-16T20:24:24.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8485e-05 2020-11-16T20:24:24.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.578999+0000", 2020-11-16T20:24:24.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9820000000000008e-06 2020-11-16T20:24:24.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.579007+0000", 2020-11-16T20:24:24.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097006060000000005 2020-11-16T20:24:24.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588707+0000", 2020-11-16T20:24:24.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010008100000000001 2020-11-16T20:24:24.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.588807+0000", 2020-11-16T20:24:24.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047694950000000003 2020-11-16T20:24:24.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.593577+0000", 2020-11-16T20:24:24.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104916 2020-11-16T20:24:24.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.593682+0000", 2020-11-16T20:24:24.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0085152460000000006 2020-11-16T20:24:24.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602197+0000", 2020-11-16T20:24:24.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8281000000000003e-05 2020-11-16T20:24:24.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.602295+0000", 2020-11-16T20:24:24.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021826139999999998 2020-11-16T20:24:24.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604478+0000", 2020-11-16T20:24:24.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025562700000000001 2020-11-16T20:24:24.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.575 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s2 31/23 ECSubWrite(tid=15, reqid=client.4444.0:85, at_version=31'7, trim_to=0'0, roll_forward_to=31'5))", 2020-11-16T20:24:24.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:24.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9159614220000005, 2020-11-16T20:24:24.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036849253999999998, 2020-11-16T20:24:24.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.576 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:24.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014849700000000001 2020-11-16T20:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605925+0000", 2020-11-16T20:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2458000000000001e-05 2020-11-16T20:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605947+0000", 2020-11-16T20:24:24.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5471999999999999e-05 2020-11-16T20:24:24.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606013+0000", 2020-11-16T20:24:24.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8510000000000001e-06 2020-11-16T20:24:24.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606020+0000", 2020-11-16T20:24:24.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000222365 2020-11-16T20:24:24.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606242+0000", 2020-11-16T20:24:24.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8460000000000003e-06 2020-11-16T20:24:24.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606252+0000", 2020-11-16T20:24:24.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036144568000000002 2020-11-16T20:24:24.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642396+0000", 2020-11-16T20:24:24.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022919699999999999 2020-11-16T20:24:24.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642626+0000", 2020-11-16T20:24:24.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036849253999999998 2020-11-16T20:24:24.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.584 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.3s1 31/23 ECSubWrite(tid=10, reqid=client.4444.0:88, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:24.584 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.613775+0000", 2020-11-16T20:24:24.584 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.907962844, 2020-11-16T20:24:24.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02647045, 2020-11-16T20:24:24.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.584 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613775+0000", 2020-11-16T20:24:24.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613775+0000", 2020-11-16T20:24:24.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021481899999999999 2020-11-16T20:24:24.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613990+0000", 2020-11-16T20:24:24.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6764e-05 2020-11-16T20:24:24.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614016+0000", 2020-11-16T20:24:24.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020432299999999999 2020-11-16T20:24:24.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614221+0000", 2020-11-16T20:24:24.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7100000000000001e-06 2020-11-16T20:24:24.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614227+0000", 2020-11-16T20:24:24.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9741999999999998e-05 2020-11-16T20:24:24.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614287+0000", 2020-11-16T20:24:24.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1365e-05 2020-11-16T20:24:24.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.614299+0000", 2020-11-16T20:24:24.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025885973 2020-11-16T20:24:24.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640185+0000", 2020-11-16T20:24:24.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0754000000000001e-05 2020-11-16T20:24:24.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640245+0000", 2020-11-16T20:24:24.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02647045 2020-11-16T20:24:24.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:112 3.5s0 3:aa220dd4:::smithi09216568-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:24.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:24.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8817177039999997, 2020-11-16T20:24:24.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033427307000000003, 2020-11-16T20:24:24.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:24.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.593 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.593 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 112 2020-11-16T20:24:24.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.593 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:24.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:24.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002370661 2020-11-16T20:24:24.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642391+0000", 2020-11-16T20:24:24.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3389999999999993e-06 2020-11-16T20:24:24.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642399+0000", 2020-11-16T20:24:24.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.401e-05 2020-11-16T20:24:24.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642413+0000", 2020-11-16T20:24:24.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7309999999999996e-06 2020-11-16T20:24:24.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642420+0000", 2020-11-16T20:24:24.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0634000000000003e-05 2020-11-16T20:24:24.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642480+0000", 2020-11-16T20:24:24.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021235000000000001 2020-11-16T20:24:24.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642693+0000", 2020-11-16T20:24:24.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010232970000000001 2020-11-16T20:24:24.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643716+0000", 2020-11-16T20:24:24.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018482157999999999 2020-11-16T20:24:24.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662198+0000", 2020-11-16T20:24:24.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011105830000000001 2020-11-16T20:24:24.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:24.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673304+0000", 2020-11-16T20:24:24.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014329699999999999 2020-11-16T20:24:24.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673447+0000", 2020-11-16T20:24:24.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033427307000000003 2020-11-16T20:24:24.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.601 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.602 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:113 3.5s0 3:aa220dd4:::smithi09216568-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:24.602 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:24.602 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.869446538, 2020-11-16T20:24:24.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035489453999999997, 2020-11-16T20:24:24.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:24.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.603 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 113 2020-11-16T20:24:24.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:24.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:24.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6527000000000003e-05 2020-11-16T20:24:24.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652348+0000", 2020-11-16T20:24:24.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7620000000000001e-06 2020-11-16T20:24:24.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652356+0000", 2020-11-16T20:24:24.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2020-11-16T20:24:24.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652371+0000", 2020-11-16T20:24:24.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0709999999999997e-06 2020-11-16T20:24:24.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652378+0000", 2020-11-16T20:24:24.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2488000000000003e-05 2020-11-16T20:24:24.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652431+0000", 2020-11-16T20:24:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107941 2020-11-16T20:24:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652539+0000", 2020-11-16T20:24:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021009645 2020-11-16T20:24:24.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673549+0000", 2020-11-16T20:24:24.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010424 2020-11-16T20:24:24.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673653+0000", 2020-11-16T20:24:24.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000577025 2020-11-16T20:24:24.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674230+0000", 2020-11-16T20:24:24.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061432429999999996 2020-11-16T20:24:24.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.680373+0000", 2020-11-16T20:24:24.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002934313 2020-11-16T20:24:24.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:24.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.683307+0000", 2020-11-16T20:24:24.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004473271 2020-11-16T20:24:24.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.687781+0000", 2020-11-16T20:24:24.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035489453999999997 2020-11-16T20:24:24.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.613 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:114 3.5s0 3:aa220dd4:::smithi09216568-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:24.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:24.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8691407009999992, 2020-11-16T20:24:24.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037793588000000003, 2020-11-16T20:24:24.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:24.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.614 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 114 2020-11-16T20:24:24.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.615 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:24.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:24.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5556000000000003e-05 2020-11-16T20:24:24.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652633+0000", 2020-11-16T20:24:24.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2020-11-16T20:24:24.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652640+0000", 2020-11-16T20:24:24.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3174e-05 2020-11-16T20:24:24.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652653+0000", 2020-11-16T20:24:24.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1580000000000002e-06 2020-11-16T20:24:24.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652659+0000", 2020-11-16T20:24:24.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.651e-05 2020-11-16T20:24:24.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652706+0000", 2020-11-16T20:24:24.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6873000000000006e-05 2020-11-16T20:24:24.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652802+0000", 2020-11-16T20:24:24.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021707194999999999 2020-11-16T20:24:24.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674510+0000", 2020-11-16T20:24:24.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114652 2020-11-16T20:24:24.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674624+0000", 2020-11-16T20:24:24.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013244752 2020-11-16T20:24:24.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.687869+0000", 2020-11-16T20:24:24.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000157813 2020-11-16T20:24:24.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688027+0000", 2020-11-16T20:24:24.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023638050000000001 2020-11-16T20:24:24.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.690391+0000", 2020-11-16T20:24:24.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037793588000000003 2020-11-16T20:24:24.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.624 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:115 3.5s0 3:ade97634:::smithi09216568-39:head [append 0~638976,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:24.624 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:24.624 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8688582950000008, 2020-11-16T20:24:24.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028710981, 2020-11-16T20:24:24.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 115 2020-11-16T20:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:24.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:24.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046393399999999999 2020-11-16T20:24:24.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653343+0000", 2020-11-16T20:24:24.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2337000000000001e-05 2020-11-16T20:24:24.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653356+0000", 2020-11-16T20:24:24.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7708000000000001e-05 2020-11-16T20:24:24.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653373+0000", 2020-11-16T20:24:24.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9990000000000008e-06 2020-11-16T20:24:24.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653381+0000", 2020-11-16T20:24:24.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2762000000000001e-05 2020-11-16T20:24:24.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653434+0000", 2020-11-16T20:24:24.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000220905 2020-11-16T20:24:24.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653655+0000", 2020-11-16T20:24:24.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001846164 2020-11-16T20:24:24.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.655501+0000", 2020-11-16T20:24:24.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007675594 2020-11-16T20:24:24.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663177+0000", 2020-11-16T20:24:24.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018319185000000002 2020-11-16T20:24:24.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:24.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681496+0000", 2020-11-16T20:24:24.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4393000000000002e-05 2020-11-16T20:24:24.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681590+0000", 2020-11-16T20:24:24.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028710981 2020-11-16T20:24:24.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.633 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:116 3.5s0 3:ade97634:::smithi09216568-39:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:24.634 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:24.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8678386739999997, 2020-11-16T20:24:24.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035068554000000002, 2020-11-16T20:24:24.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:24.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.635 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 116 2020-11-16T20:24:24.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.635 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:24.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:24.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6311999999999999e-05 2020-11-16T20:24:24.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653955+0000", 2020-11-16T20:24:24.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9060000000000008e-06 2020-11-16T20:24:24.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653964+0000", 2020-11-16T20:24:24.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3818000000000001e-05 2020-11-16T20:24:24.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653978+0000", 2020-11-16T20:24:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.755e-06 2020-11-16T20:24:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653984+0000", 2020-11-16T20:24:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079596330000000007 2020-11-16T20:24:24.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.661943+0000", 2020-11-16T20:24:24.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012344799999999999 2020-11-16T20:24:24.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662067+0000", 2020-11-16T20:24:24.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019626787999999999 2020-11-16T20:24:24.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681694+0000", 2020-11-16T20:24:24.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107509 2020-11-16T20:24:24.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681801+0000", 2020-11-16T20:24:24.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000509633 2020-11-16T20:24:24.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682311+0000", 2020-11-16T20:24:24.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061688259999999996 2020-11-16T20:24:24.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688480+0000", 2020-11-16T20:24:24.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4555999999999995e-05 2020-11-16T20:24:24.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:24.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688564+0000", 2020-11-16T20:24:24.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040337000000000001 2020-11-16T20:24:24.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688968+0000", 2020-11-16T20:24:24.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035068554000000002 2020-11-16T20:24:24.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.645 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.645 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:117 3.5s0 3:ade97634:::smithi09216568-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:24.646 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:24.646 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8677079879999994, 2020-11-16T20:24:24.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037529909, 2020-11-16T20:24:24.646 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.646 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:24.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 117 2020-11-16T20:24:24.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.647 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:24.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:24.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7752e-05 2020-11-16T20:24:24.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654067+0000", 2020-11-16T20:24:24.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8970000000000002e-06 2020-11-16T20:24:24.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654074+0000", 2020-11-16T20:24:24.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2906e-05 2020-11-16T20:24:24.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654087+0000", 2020-11-16T20:24:24.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.181e-06 2020-11-16T20:24:24.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654092+0000", 2020-11-16T20:24:24.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080038750000000006 2020-11-16T20:24:24.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662096+0000", 2020-11-16T20:24:24.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7354000000000003e-05 2020-11-16T20:24:24.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662194+0000", 2020-11-16T20:24:24.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020391761000000001 2020-11-16T20:24:24.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682585+0000", 2020-11-16T20:24:24.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114128 2020-11-16T20:24:24.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682700+0000", 2020-11-16T20:24:24.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063062220000000002 2020-11-16T20:24:24.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689006+0000", 2020-11-16T20:24:24.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5025000000000007e-05 2020-11-16T20:24:24.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689091+0000", 2020-11-16T20:24:24.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024688079999999999 2020-11-16T20:24:24.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.691560+0000", 2020-11-16T20:24:24.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037529909 2020-11-16T20:24:24.655 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.656 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 31/26 ECSubWrite(tid=8, reqid=client.4444.0:110, at_version=31'4, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:24.656 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:24.656 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8596816870000001, 2020-11-16T20:24:24.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032919114999999999, 2020-11-16T20:24:24.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:24.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:24.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6403999999999999e-05 2020-11-16T20:24:24.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662092+0000", 2020-11-16T20:24:24.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2906999999999999e-05 2020-11-16T20:24:24.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662115+0000", 2020-11-16T20:24:24.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027791066999999999 2020-11-16T20:24:24.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689906+0000", 2020-11-16T20:24:24.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8580000000000002e-06 2020-11-16T20:24:24.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689913+0000", 2020-11-16T20:24:24.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0863999999999999e-05 2020-11-16T20:24:24.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689974+0000", 2020-11-16T20:24:24.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4901e-05 2020-11-16T20:24:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689989+0000", 2020-11-16T20:24:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049304020000000004 2020-11-16T20:24:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.694919+0000", 2020-11-16T20:24:24.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5711999999999998e-05 2020-11-16T20:24:24.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.694975+0000", 2020-11-16T20:24:24.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032919114999999999 2020-11-16T20:24:24.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.664 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.664 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 31/26 ECSubWrite(tid=10, reqid=client.4444.0:136, at_version=31'5, trim_to=0'0, roll_forward_to=31'3))", 2020-11-16T20:24:24.664 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:24.664 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8324058920000006, 2020-11-16T20:24:24.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042429041000000001, 2020-11-16T20:24:24.665 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.665 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:24.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:24.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050619800000000002 2020-11-16T20:24:24.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689838+0000", 2020-11-16T20:24:24.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8632e-05 2020-11-16T20:24:24.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689867+0000", 2020-11-16T20:24:24.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03374162 2020-11-16T20:24:24.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723608+0000", 2020-11-16T20:24:24.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5469999999999999e-06 2020-11-16T20:24:24.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723616+0000", 2020-11-16T20:24:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8924000000000006e-05 2020-11-16T20:24:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723685+0000", 2020-11-16T20:24:24.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7054e-05 2020-11-16T20:24:24.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723702+0000", 2020-11-16T20:24:24.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080210799999999999 2020-11-16T20:24:24.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.731723+0000", 2020-11-16T20:24:24.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7985999999999998e-05 2020-11-16T20:24:24.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.731761+0000", 2020-11-16T20:24:24.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042429041000000001 2020-11-16T20:24:24.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.672 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.2s1 31/23 ECSubRead(tid=47, to_read={3:47d6be26:::smithi09216568-44:head=0,4096,0}, subchunks={3:47d6be26:::smithi09216568-44:head=[0,1]}, attrs_to_read=))", 2020-11-16T20:24:24.672 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:24.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8048918100000009, 2020-11-16T20:24:24.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027734008000000001, 2020-11-16T20:24:24.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:24.673 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8687999999999998e-05 2020-11-16T20:24:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716905+0000", 2020-11-16T20:24:24.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0064999999999999e-05 2020-11-16T20:24:24.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716915+0000", 2020-11-16T20:24:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026392019999999999 2020-11-16T20:24:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743307+0000", 2020-11-16T20:24:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1610999999999999e-05 2020-11-16T20:24:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743318+0000", 2020-11-16T20:24:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4503000000000001e-05 2020-11-16T20:24:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743393+0000", 2020-11-16T20:24:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001187121 2020-11-16T20:24:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.744580+0000", 2020-11-16T20:24:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027734008000000001 2020-11-16T20:24:24.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:185 3.es0 3:746783df:::smithi09216568-20:head [append 0~507904] snapc 4=[4,3,2] ondisk+write+known_if_redirected e34)", 2020-11-16T20:24:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:18.576813+0000", 2020-11-16T20:24:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.944924801, 2020-11-16T20:24:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032237632000000002, 2020-11-16T20:24:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 185 2020-11-16T20:24:24.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.576813+0000", 2020-11-16T20:24:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.576813+0000", 2020-11-16T20:24:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036412800000000002 2020-11-16T20:24:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577177+0000", 2020-11-16T20:24:24.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0089999999999999e-05 2020-11-16T20:24:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577197+0000", 2020-11-16T20:24:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2817000000000001e-05 2020-11-16T20:24:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577230+0000", 2020-11-16T20:24:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7073000000000001e-05 2020-11-16T20:24:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577247+0000", 2020-11-16T20:24:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013625000000000001 2020-11-16T20:24:24.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577383+0000", 2020-11-16T20:24:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6784999999999999e-05 2020-11-16T20:24:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2020-11-16T20:24:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577400+0000", 2020-11-16T20:24:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003451974 2020-11-16T20:24:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580852+0000", 2020-11-16T20:24:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012214800000000001 2020-11-16T20:24:24.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580974+0000", 2020-11-16T20:24:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001202928 2020-11-16T20:24:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.582177+0000", 2020-11-16T20:24:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018978376000000002 2020-11-16T20:24:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.601155+0000", 2020-11-16T20:24:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077913540000000003 2020-11-16T20:24:24.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.608947+0000", 2020-11-16T20:24:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010370900000000001 2020-11-16T20:24:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.609051+0000", 2020-11-16T20:24:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032237632000000002 2020-11-16T20:24:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.690 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:186 3.es0 3:746783df:::smithi09216568-20:head [append 0~401408] snapc 4=[4,3,2] ondisk+write+known_if_redirected e34)", 2020-11-16T20:24:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:18.579729+0000", 2020-11-16T20:24:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9420083039999998, 2020-11-16T20:24:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031962467000000001, 2020-11-16T20:24:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.691 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 186 2020-11-16T20:24:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.579729+0000", 2020-11-16T20:24:24.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.579729+0000", 2020-11-16T20:24:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062883899999999996 2020-11-16T20:24:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580358+0000", 2020-11-16T20:24:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8054999999999999e-05 2020-11-16T20:24:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580376+0000", 2020-11-16T20:24:24.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9548000000000002e-05 2020-11-16T20:24:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580416+0000", 2020-11-16T20:24:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4072e-05 2020-11-16T20:24:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580430+0000", 2020-11-16T20:24:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013003321999999999 2020-11-16T20:24:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.593433+0000", 2020-11-16T20:24:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021100700000000001 2020-11-16T20:24:24.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.593644+0000", 2020-11-16T20:24:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000748498 2020-11-16T20:24:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.594393+0000", 2020-11-16T20:24:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082229850000000004 2020-11-16T20:24:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.602616+0000", 2020-11-16T20:24:24.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089692920000000002 2020-11-16T20:24:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.611585+0000", 2020-11-16T20:24:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010684900000000001 2020-11-16T20:24:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.611692+0000", 2020-11-16T20:24:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031962467000000001 2020-11-16T20:24:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:187 3.es0 3:746783df:::smithi09216568-20:head [setxattr _header (56) in=7b] snapc 4=[4,3,2] ondisk+write+known_if_redirected e34)", 2020-11-16T20:24:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:18.580531+0000", 2020-11-16T20:24:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9412067039999998, 2020-11-16T20:24:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82514190300000001, 2020-11-16T20:24:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 187 2020-11-16T20:24:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580531+0000", 2020-11-16T20:24:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580531+0000", 2020-11-16T20:24:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133392 2020-11-16T20:24:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580664+0000", 2020-11-16T20:24:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6212e-05 2020-11-16T20:24:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580681+0000", 2020-11-16T20:24:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9286e-05 2020-11-16T20:24:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580710+0000", 2020-11-16T20:24:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4297000000000001e-05 2020-11-16T20:24:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580724+0000", 2020-11-16T20:24:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019066024000000001 2020-11-16T20:24:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.599790+0000", 2020-11-16T20:24:24.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112787 2020-11-16T20:24:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.599903+0000", 2020-11-16T20:24:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00065871699999999996 2020-11-16T20:24:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.600562+0000", 2020-11-16T20:24:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047374749999999997 2020-11-16T20:24:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.605299+0000", 2020-11-16T20:24:24.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80021618900000002 2020-11-16T20:24:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.405515+0000", 2020-11-16T20:24:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000157524 2020-11-16T20:24:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.405673+0000", 2020-11-16T20:24:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82514190300000001 2020-11-16T20:24:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:188 3.es0 3:746783df:::smithi09216568-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", 2020-11-16T20:24:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:18.580843+0000", 2020-11-16T20:24:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9408945769999999, 2020-11-16T20:24:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.828529878, 2020-11-16T20:24:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:24.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 188 2020-11-16T20:24:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:24.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580843+0000", 2020-11-16T20:24:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580843+0000", 2020-11-16T20:24:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1105000000000003e-05 2020-11-16T20:24:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580934+0000", 2020-11-16T20:24:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4115e-05 2020-11-16T20:24:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:24.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580948+0000", 2020-11-16T20:24:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8566999999999999e-05 2020-11-16T20:24:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580977+0000", 2020-11-16T20:24:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3980000000000005e-06 2020-11-16T20:24:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580986+0000", 2020-11-16T20:24:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019924358 2020-11-16T20:24:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.600911+0000", 2020-11-16T20:24:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015903799999999999 2020-11-16T20:24:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.601070+0000", 2020-11-16T20:24:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80550670499999999 2020-11-16T20:24:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.406576+0000", 2020-11-16T20:24:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169784 2020-11-16T20:24:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.406746+0000", 2020-11-16T20:24:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002626808 2020-11-16T20:24:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.409373+0000", 2020-11-16T20:24:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.828529878 2020-11-16T20:24:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.719 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:24.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:24.720 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:24.722 INFO:tasks.rados.rados.0.smithi092.stdout:104: done (3 left) 2020-11-16T20:24:24.722 INFO:tasks.rados.rados.0.smithi092.stdout:105: delete oid 2 current snap is 5 2020-11-16T20:24:24.723 INFO:tasks.rados.rados.0.smithi092.stdout:101: finishing write tid 4 to smithi09216568-11 2020-11-16T20:24:24.723 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 31 (ObjNum 59 snap 5 seq_num 59) dirty exists 2020-11-16T20:24:24.723 INFO:tasks.rados.rados.0.smithi092.stdout:101: left oid 11 (ObjNum 59 snap 5 seq_num 59) 2020-11-16T20:24:24.723 INFO:tasks.rados.rados.0.smithi092.stdout:103: finishing rollback tid 0 to smithi09216568-8 2020-11-16T20:24:24.723 INFO:tasks.rados.rados.0.smithi092.stdout:103: finishing rollback tid 1 to smithi09216568-8 2020-11-16T20:24:24.723 INFO:tasks.rados.rados.0.smithi092.stdout:103: finishing rollback tid 2 to smithi09216568-8 2020-11-16T20:24:24.723 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 8 v 16 (ObjNum 7 snap 0 seq_num 7) dirty exists 2020-11-16T20:24:24.724 INFO:tasks.rados.rados.0.smithi092.stdout:97: finishing write tid 3 to smithi09216568-38 2020-11-16T20:24:24.724 INFO:tasks.rados.rados.0.smithi092.stdout:97: finishing write tid 4 to smithi09216568-38 2020-11-16T20:24:24.724 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 38 v 18 (ObjNum 58 snap 5 seq_num 58) dirty exists 2020-11-16T20:24:24.724 INFO:tasks.rados.rados.0.smithi092.stdout:97: left oid 38 (ObjNum 58 snap 5 seq_num 58) 2020-11-16T20:24:24.724 INFO:tasks.rados.rados.0.smithi092.stdout:97: done (3 left) 2020-11-16T20:24:24.724 INFO:tasks.rados.rados.0.smithi092.stdout:101: done (2 left) 2020-11-16T20:24:24.724 INFO:tasks.rados.rados.0.smithi092.stdout:103: done (1 left) 2020-11-16T20:24:24.724 INFO:tasks.rados.rados.0.smithi092.stdout:105: done (0 left) 2020-11-16T20:24:24.725 INFO:tasks.rados.rados.0.smithi092.stdout:106: read oid 37 snap -1 2020-11-16T20:24:24.725 INFO:tasks.rados.rados.0.smithi092.stdout:106: expect (ObjNum 57 snap 4 seq_num 57) 2020-11-16T20:24:24.725 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 8 current snap is 5 2020-11-16T20:24:24.725 INFO:tasks.rados.rados.0.smithi092.stdout:107: seq_num 61 ranges {122880=630784,753664=401408} 2020-11-16T20:24:24.725 INFO:tasks.rados.rados.0.smithi092.stdout:107: writing smithi09216568-8 from 122880 to 753664 tid 1 2020-11-16T20:24:24.725 INFO:tasks.rados.rados.0.smithi092.stdout:107: writing smithi09216568-8 from 753664 to 1155072 tid 2 2020-11-16T20:24:24.725 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 50 current snap is 5 2020-11-16T20:24:24.725 INFO:tasks.rados.rados.0.smithi092.stdout:108: seq_num 62 ranges {827392=647168,1474560=401408} 2020-11-16T20:24:24.726 INFO:tasks.rados.rados.0.smithi092.stdout:108: writing smithi09216568-50 from 827392 to 1474560 tid 1 2020-11-16T20:24:24.726 INFO:tasks.rados.rados.0.smithi092.stdout:108: writing smithi09216568-50 from 1474560 to 1875968 tid 2 2020-11-16T20:24:24.726 INFO:tasks.rados.rados.0.smithi092.stdout:109: snap_remove snap 4 2020-11-16T20:24:24.726 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2020-11-16T20:24:24.726 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2020-11-16T20:24:24.727 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2020-11-16T20:24:24.727 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:24.721+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:24.728 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-11-16T20:24:24.822 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:24.821+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:24.883 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:24.883 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:24.893 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-11-16T20:24:24.923 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:24.920+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:25.023 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:25.024+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:25.052 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:25.052 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:25.052 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:25.052 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:25.053 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:25.062 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-11-16T20:24:25.124 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:25.124+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:25.181 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:25.181 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:25.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:25.182 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:25.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.182 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:14 3.9s0 3:910fffa8:::smithi09216568-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-11-16T20:24:25.182 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:25.182 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6488433879999995, 2020-11-16T20:24:25.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033104548999999997, 2020-11-16T20:24:25.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 14 2020-11-16T20:24:25.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:25.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:25.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8910999999999999e-05 2020-11-16T20:24:25.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532325+0000", 2020-11-16T20:24:25.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2323e-05 2020-11-16T20:24:25.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532347+0000", 2020-11-16T20:24:25.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7090999999999999e-05 2020-11-16T20:24:25.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532374+0000", 2020-11-16T20:24:25.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5399999999999998e-06 2020-11-16T20:24:25.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532382+0000", 2020-11-16T20:24:25.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077685499999999999 2020-11-16T20:24:25.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540150+0000", 2020-11-16T20:24:25.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119175 2020-11-16T20:24:25.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540269+0000", 2020-11-16T20:24:25.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011674891 2020-11-16T20:24:25.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551944+0000", 2020-11-16T20:24:25.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010398199999999999 2020-11-16T20:24:25.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.552048+0000", 2020-11-16T20:24:25.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077962600000000002 2020-11-16T20:24:25.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.559845+0000", 2020-11-16T20:24:25.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028470140000000001 2020-11-16T20:24:25.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562692+0000", 2020-11-16T20:24:25.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002587677 2020-11-16T20:24:25.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565279+0000", 2020-11-16T20:24:25.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1135000000000001e-05 2020-11-16T20:24:25.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565370+0000", 2020-11-16T20:24:25.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033104548999999997 2020-11-16T20:24:25.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.194 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:15 3.9s0 3:910fffa8:::smithi09216568-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e30)", 2020-11-16T20:24:25.194 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:25.194 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6486757910000005, 2020-11-16T20:24:25.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037521408999999999, 2020-11-16T20:24:25.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.194 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:25.194 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.195 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.195 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15 2020-11-16T20:24:25.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.195 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:25.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:25.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9020000000000001e-05 2020-11-16T20:24:25.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532493+0000", 2020-11-16T20:24:25.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0396e-05 2020-11-16T20:24:25.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532503+0000", 2020-11-16T20:24:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9660000000000002e-05 2020-11-16T20:24:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532523+0000", 2020-11-16T20:24:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8280000000000003e-06 2020-11-16T20:24:25.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532530+0000", 2020-11-16T20:24:25.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077981070000000003 2020-11-16T20:24:25.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540329+0000", 2020-11-16T20:24:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8363000000000007e-05 2020-11-16T20:24:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540427+0000", 2020-11-16T20:24:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019650447000000001 2020-11-16T20:24:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560077+0000", 2020-11-16T20:24:25.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2565999999999994e-05 2020-11-16T20:24:25.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560170+0000", 2020-11-16T20:24:25.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052459309999999997 2020-11-16T20:24:25.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565416+0000", 2020-11-16T20:24:25.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1088999999999995e-05 2020-11-16T20:24:25.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565507+0000", 2020-11-16T20:24:25.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044480020000000004 2020-11-16T20:24:25.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.569955+0000", 2020-11-16T20:24:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037521408999999999 2020-11-16T20:24:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.204 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:23 3.ds0 3:b099cead:::smithi09216568-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-11-16T20:24:25.205 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:25.205 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6482314720000009, 2020-11-16T20:24:25.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025302294999999999, 2020-11-16T20:24:25.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.206 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.206 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23 2020-11-16T20:24:25.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.206 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:25.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:25.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5377999999999997e-05 2020-11-16T20:24:25.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532933+0000", 2020-11-16T20:24:25.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9703000000000001e-05 2020-11-16T20:24:25.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532953+0000", 2020-11-16T20:24:25.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2229e-05 2020-11-16T20:24:25.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532975+0000", 2020-11-16T20:24:25.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041310599999999998 2020-11-16T20:24:25.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537106+0000", 2020-11-16T20:24:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055730839999999999 2020-11-16T20:24:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542679+0000", 2020-11-16T20:24:25.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013964700000000001 2020-11-16T20:24:25.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542819+0000", 2020-11-16T20:24:25.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050541550000000003 2020-11-16T20:24:25.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.547873+0000", 2020-11-16T20:24:25.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253999999999993e-05 2020-11-16T20:24:25.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.547934+0000", 2020-11-16T20:24:25.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000519886 2020-11-16T20:24:25.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548454+0000", 2020-11-16T20:24:25.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066483360000000003 2020-11-16T20:24:25.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.555103+0000", 2020-11-16T20:24:25.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026224410000000001 2020-11-16T20:24:25.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557725+0000", 2020-11-16T20:24:25.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045512200000000002 2020-11-16T20:24:25.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558180+0000", 2020-11-16T20:24:25.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025302294999999999 2020-11-16T20:24:25.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.215 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.216 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:24 3.ds0 3:b099cead:::smithi09216568-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e30)", 2020-11-16T20:24:25.216 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:25.216 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6439375340000009, 2020-11-16T20:24:25.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025939752999999999, 2020-11-16T20:24:25.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:25.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.217 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 24 2020-11-16T20:24:25.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.217 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:25.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8302e-05 2020-11-16T20:24:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537210+0000", 2020-11-16T20:24:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5029999999999997e-06 2020-11-16T20:24:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537219+0000", 2020-11-16T20:24:25.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5136e-05 2020-11-16T20:24:25.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537234+0000", 2020-11-16T20:24:25.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2160000000000003e-06 2020-11-16T20:24:25.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537240+0000", 2020-11-16T20:24:25.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056701290000000003 2020-11-16T20:24:25.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542910+0000", 2020-11-16T20:24:25.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011685599999999999 2020-11-16T20:24:25.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543027+0000", 2020-11-16T20:24:25.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056655689999999996 2020-11-16T20:24:25.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548692+0000", 2020-11-16T20:24:25.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9920000000000003e-05 2020-11-16T20:24:25.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548742+0000", 2020-11-16T20:24:25.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094867259999999991 2020-11-16T20:24:25.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558229+0000", 2020-11-16T20:24:25.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8435000000000003e-05 2020-11-16T20:24:25.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558318+0000", 2020-11-16T20:24:25.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047939610000000002 2020-11-16T20:24:25.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563111+0000", 2020-11-16T20:24:25.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025939752999999999 2020-11-16T20:24:25.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.226 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.227 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:32 3.9s0 3:94376c81:::smithi09216568-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.227 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:25.227 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6415470719999998, 2020-11-16T20:24:25.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026430479999999999, 2020-11-16T20:24:25.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.228 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32 2020-11-16T20:24:25.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.228 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:25.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5041999999999999e-05 2020-11-16T20:24:25.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539617+0000", 2020-11-16T20:24:25.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2020-11-16T20:24:25.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539624+0000", 2020-11-16T20:24:25.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2020-11-16T20:24:25.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539636+0000", 2020-11-16T20:24:25.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.075e-06 2020-11-16T20:24:25.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539643+0000", 2020-11-16T20:24:25.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039536490000000001 2020-11-16T20:24:25.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543597+0000", 2020-11-16T20:24:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112988 2020-11-16T20:24:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543710+0000", 2020-11-16T20:24:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016926085 2020-11-16T20:24:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560636+0000", 2020-11-16T20:24:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8800000000000004e-05 2020-11-16T20:24:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560725+0000", 2020-11-16T20:24:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00080697699999999995 2020-11-16T20:24:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.561532+0000", 2020-11-16T20:24:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042868910000000001 2020-11-16T20:24:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565818+0000", 2020-11-16T20:24:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5162999999999998e-05 2020-11-16T20:24:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565904+0000", 2020-11-16T20:24:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9067999999999995e-05 2020-11-16T20:24:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565993+0000", 2020-11-16T20:24:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026430479999999999 2020-11-16T20:24:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:33 3.9s0 3:94376c81:::smithi09216568-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6414252779999998, 2020-11-16T20:24:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030497335, 2020-11-16T20:24:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33 2020-11-16T20:24:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8398000000000001e-05 2020-11-16T20:24:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539722+0000", 2020-11-16T20:24:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.993e-06 2020-11-16T20:24:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539728+0000", 2020-11-16T20:24:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2140000000000001e-05 2020-11-16T20:24:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539741+0000", 2020-11-16T20:24:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9979999999999997e-06 2020-11-16T20:24:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539745+0000", 2020-11-16T20:24:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004070204 2020-11-16T20:24:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543815+0000", 2020-11-16T20:24:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535999999999999 2020-11-16T20:24:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543920+0000", 2020-11-16T20:24:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018410834000000001 2020-11-16T20:24:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562331+0000", 2020-11-16T20:24:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4544999999999997e-05 2020-11-16T20:24:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562425+0000", 2020-11-16T20:24:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035976329999999998 2020-11-16T20:24:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.566023+0000", 2020-11-16T20:24:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2955999999999996e-05 2020-11-16T20:24:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.566116+0000", 2020-11-16T20:24:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040652739999999998 2020-11-16T20:24:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.570181+0000", 2020-11-16T20:24:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030497335 2020-11-16T20:24:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:35 3.9s0 3:9cc22e43:::smithi09216568-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6402527310000004, 2020-11-16T20:24:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11108446600000001, 2020-11-16T20:24:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 35 2020-11-16T20:24:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4415e-05 2020-11-16T20:24:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540911+0000", 2020-11-16T20:24:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.533e-06 2020-11-16T20:24:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540917+0000", 2020-11-16T20:24:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-05 2020-11-16T20:24:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540930+0000", 2020-11-16T20:24:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2769999999999999e-06 2020-11-16T20:24:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540934+0000", 2020-11-16T20:24:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010586527 2020-11-16T20:24:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551521+0000", 2020-11-16T20:24:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012027399999999999 2020-11-16T20:24:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551641+0000", 2020-11-16T20:24:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011083160999999999 2020-11-16T20:24:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562724+0000", 2020-11-16T20:24:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4109000000000004e-05 2020-11-16T20:24:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562808+0000", 2020-11-16T20:24:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000459751 2020-11-16T20:24:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563268+0000", 2020-11-16T20:24:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060973590000000001 2020-11-16T20:24:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.569365+0000", 2020-11-16T20:24:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082458486999999997 2020-11-16T20:24:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651824+0000", 2020-11-16T20:24:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117173 2020-11-16T20:24:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651941+0000", 2020-11-16T20:24:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11108446600000001 2020-11-16T20:24:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:36 3.9s0 3:9cc22e43:::smithi09216568-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6401336579999999, 2020-11-16T20:24:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.116868257, 2020-11-16T20:24:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36 2020-11-16T20:24:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9727000000000001e-05 2020-11-16T20:24:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541015+0000", 2020-11-16T20:24:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1739999999999997e-06 2020-11-16T20:24:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541022+0000", 2020-11-16T20:24:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1854e-05 2020-11-16T20:24:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541033+0000", 2020-11-16T20:24:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.799e-06 2020-11-16T20:24:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541037+0000", 2020-11-16T20:24:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019161628999999999 2020-11-16T20:24:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560199+0000", 2020-11-16T20:24:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102256 2020-11-16T20:24:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560301+0000", 2020-11-16T20:24:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032318479999999998 2020-11-16T20:24:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563533+0000", 2020-11-16T20:24:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2615999999999996e-05 2020-11-16T20:24:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563626+0000", 2020-11-16T20:24:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088372749 2020-11-16T20:24:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651998+0000", 2020-11-16T20:24:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113002 2020-11-16T20:24:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652111+0000", 2020-11-16T20:24:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057326030000000002 2020-11-16T20:24:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.657844+0000", 2020-11-16T20:24:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.116868257 2020-11-16T20:24:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=10, last_complete=31'6, committed=1, applied=1))", 2020-11-16T20:24:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6154918820000006, 2020-11-16T20:24:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086340027, 2020-11-16T20:24:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6184999999999999e-05 2020-11-16T20:24:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565664+0000", 2020-11-16T20:24:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7180000000000002e-06 2020-11-16T20:24:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565672+0000", 2020-11-16T20:24:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085916762999999993 2020-11-16T20:24:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651589+0000", 2020-11-16T20:24:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5119999999999997e-06 2020-11-16T20:24:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651596+0000", 2020-11-16T20:24:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3547000000000001e-05 2020-11-16T20:24:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651649+0000", 2020-11-16T20:24:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030830199999999999 2020-11-16T20:24:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651957+0000", 2020-11-16T20:24:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086340027 2020-11-16T20:24:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:85 3.9s0 3:9bc9395c:::smithi09216568-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5773930909999994, 2020-11-16T20:24:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048890797999999999, 2020-11-16T20:24:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 85 2020-11-16T20:24:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033801700000000003 2020-11-16T20:24:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604054+0000", 2020-11-16T20:24:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1904e-05 2020-11-16T20:24:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604066+0000", 2020-11-16T20:24:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2374e-05 2020-11-16T20:24:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604088+0000", 2020-11-16T20:24:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0937e-05 2020-11-16T20:24:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604099+0000", 2020-11-16T20:24:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6842999999999996e-05 2020-11-16T20:24:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604176+0000", 2020-11-16T20:24:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180261 2020-11-16T20:24:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604357+0000", 2020-11-16T20:24:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012281449999999999 2020-11-16T20:24:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605585+0000", 2020-11-16T20:24:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074810600000000003 2020-11-16T20:24:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613066+0000", 2020-11-16T20:24:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039419787999999997 2020-11-16T20:24:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652486+0000", 2020-11-16T20:24:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121469 2020-11-16T20:24:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652607+0000", 2020-11-16T20:24:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048890797999999999 2020-11-16T20:24:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:86 3.9s0 3:9bc9395c:::smithi09216568-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5769514109999996, 2020-11-16T20:24:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071173710000000001, 2020-11-16T20:24:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 86 2020-11-16T20:24:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7161999999999999e-05 2020-11-16T20:24:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604215+0000", 2020-11-16T20:24:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3989999999999998e-06 2020-11-16T20:24:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604222+0000", 2020-11-16T20:24:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3733999999999999e-05 2020-11-16T20:24:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604236+0000", 2020-11-16T20:24:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3090000000000003e-06 2020-11-16T20:24:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604241+0000", 2020-11-16T20:24:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074329820000000003 2020-11-16T20:24:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611674+0000", 2020-11-16T20:24:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117886 2020-11-16T20:24:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611792+0000", 2020-11-16T20:24:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040893269000000003 2020-11-16T20:24:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652686+0000", 2020-11-16T20:24:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9841999999999998e-05 2020-11-16T20:24:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652775+0000", 2020-11-16T20:24:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052300899999999995 2020-11-16T20:24:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653298+0000", 2020-11-16T20:24:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089837800000000002 2020-11-16T20:24:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654197+0000", 2020-11-16T20:24:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021031319999999999 2020-11-16T20:24:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675228+0000", 2020-11-16T20:24:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010342 2020-11-16T20:24:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675332+0000", 2020-11-16T20:24:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071173710000000001 2020-11-16T20:24:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:87 3.9s0 3:9bc9395c:::smithi09216568-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5768205829999999, 2020-11-16T20:24:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072859264000000007, 2020-11-16T20:24:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 87 2020-11-16T20:24:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1205000000000003e-05 2020-11-16T20:24:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604330+0000", 2020-11-16T20:24:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7689999999999999e-06 2020-11-16T20:24:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604337+0000", 2020-11-16T20:24:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3181e-05 2020-11-16T20:24:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604350+0000", 2020-11-16T20:24:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1000000000000003e-06 2020-11-16T20:24:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604355+0000", 2020-11-16T20:24:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074779859999999998 2020-11-16T20:24:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611833+0000", 2020-11-16T20:24:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087081400000000003 2020-11-16T20:24:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612704+0000", 2020-11-16T20:24:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040880316 2020-11-16T20:24:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653584+0000", 2020-11-16T20:24:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5697000000000003e-05 2020-11-16T20:24:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653670+0000", 2020-11-16T20:24:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021708466999999999 2020-11-16T20:24:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675378+0000", 2020-11-16T20:24:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7913999999999998e-05 2020-11-16T20:24:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675476+0000", 2020-11-16T20:24:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016718150000000001 2020-11-16T20:24:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.677148+0000", 2020-11-16T20:24:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072859264000000007 2020-11-16T20:24:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:103 3.9s0 3:98ee58dd:::smithi09216568-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5551821480000001, 2020-11-16T20:24:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037777976999999997, 2020-11-16T20:24:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 103 2020-11-16T20:24:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000376745 2020-11-16T20:24:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626304+0000", 2020-11-16T20:24:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0266e-05 2020-11-16T20:24:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626314+0000", 2020-11-16T20:24:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327e-05 2020-11-16T20:24:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626334+0000", 2020-11-16T20:24:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0482999999999999e-05 2020-11-16T20:24:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626345+0000", 2020-11-16T20:24:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7775999999999996e-05 2020-11-16T20:24:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626413+0000", 2020-11-16T20:24:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020824099999999999 2020-11-16T20:24:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626621+0000", 2020-11-16T20:24:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097088399999999998 2020-11-16T20:24:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.627592+0000", 2020-11-16T20:24:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074737739999999999 2020-11-16T20:24:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635066+0000", 2020-11-16T20:24:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028500588 2020-11-16T20:24:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663566+0000", 2020-11-16T20:24:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013889300000000001 2020-11-16T20:24:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663705+0000", 2020-11-16T20:24:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037777976999999997 2020-11-16T20:24:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:104 3.9s0 3:98ee58dd:::smithi09216568-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5547084580000003, 2020-11-16T20:24:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109118772, 2020-11-16T20:24:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104 2020-11-16T20:24:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4968999999999997e-05 2020-11-16T20:24:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626436+0000", 2020-11-16T20:24:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.33e-06 2020-11-16T20:24:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626440+0000", 2020-11-16T20:24:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1780000000000003e-06 2020-11-16T20:24:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626448+0000", 2020-11-16T20:24:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5559999999999998e-06 2020-11-16T20:24:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626454+0000", 2020-11-16T20:24:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082374119999999995 2020-11-16T20:24:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634691+0000", 2020-11-16T20:24:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6558999999999995e-05 2020-11-16T20:24:25.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634788+0000", 2020-11-16T20:24:25.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029028163999999999 2020-11-16T20:24:25.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663816+0000", 2020-11-16T20:24:25.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9767999999999998e-05 2020-11-16T20:24:25.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663906+0000", 2020-11-16T20:24:25.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088046700000000001 2020-11-16T20:24:25.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.664786+0000", 2020-11-16T20:24:25.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001328255 2020-11-16T20:24:25.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.666114+0000", 2020-11-16T20:24:25.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069334706999999995 2020-11-16T20:24:25.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735449+0000", 2020-11-16T20:24:25.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0407000000000005e-05 2020-11-16T20:24:25.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735520+0000", 2020-11-16T20:24:25.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109118772 2020-11-16T20:24:25.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.333 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:105 3.9s0 3:98ee58dd:::smithi09216568-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:25.334 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5546092680000001, 2020-11-16T20:24:25.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11144889600000001, 2020-11-16T20:24:25.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:25.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2020-11-16T20:24:25.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.335 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:25.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:25.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0299000000000002e-05 2020-11-16T20:24:25.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626540+0000", 2020-11-16T20:24:25.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3050000000000004e-06 2020-11-16T20:24:25.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626548+0000", 2020-11-16T20:24:25.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.878e-05 2020-11-16T20:24:25.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626566+0000", 2020-11-16T20:24:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3839999999999996e-06 2020-11-16T20:24:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626572+0000", 2020-11-16T20:24:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082488189999999992 2020-11-16T20:24:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634821+0000", 2020-11-16T20:24:25.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4552e-05 2020-11-16T20:24:25.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634905+0000", 2020-11-16T20:24:25.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030415658000000002 2020-11-16T20:24:25.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.665321+0000", 2020-11-16T20:24:25.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0865000000000005e-05 2020-11-16T20:24:25.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.665412+0000", 2020-11-16T20:24:25.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.070142494 2020-11-16T20:24:25.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735554+0000", 2020-11-16T20:24:25.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9005000000000007e-05 2020-11-16T20:24:25.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735633+0000", 2020-11-16T20:24:25.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023157350000000002 2020-11-16T20:24:25.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.737949+0000", 2020-11-16T20:24:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11144889600000001 2020-11-16T20:24:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.345 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:108 3.as0 3:5d07813d:::smithi09216568-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.345 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:25.345 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5501810309999993, 2020-11-16T20:24:25.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024149034999999999, 2020-11-16T20:24:25.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:25.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.346 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.346 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 108 2020-11-16T20:24:25.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:25.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:25.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4607999999999997e-05 2020-11-16T20:24:25.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630973+0000", 2020-11-16T20:24:25.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4680000000000003e-06 2020-11-16T20:24:25.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630980+0000", 2020-11-16T20:24:25.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019917399999999999 2020-11-16T20:24:25.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631179+0000", 2020-11-16T20:24:25.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4740000000000002e-06 2020-11-16T20:24:25.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631186+0000", 2020-11-16T20:24:25.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010229520000000001 2020-11-16T20:24:25.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641415+0000", 2020-11-16T20:24:25.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3351999999999997e-05 2020-11-16T20:24:25.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641499+0000", 2020-11-16T20:24:25.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074322040000000004 2020-11-16T20:24:25.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.648931+0000", 2020-11-16T20:24:25.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.166e-05 2020-11-16T20:24:25.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.649023+0000", 2020-11-16T20:24:25.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039121529999999998 2020-11-16T20:24:25.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652935+0000", 2020-11-16T20:24:25.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140617 2020-11-16T20:24:25.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653075+0000", 2020-11-16T20:24:25.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020018050000000002 2020-11-16T20:24:25.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.655077+0000", 2020-11-16T20:24:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024149034999999999 2020-11-16T20:24:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=24, last_complete=31'10, committed=1, applied=1))", 2020-11-16T20:24:25.355 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:25.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.5061242910000008, 2020-11-16T20:24:25.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060547953000000002, 2020-11-16T20:24:25.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:25.356 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6891000000000003e-05 2020-11-16T20:24:25.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675022+0000", 2020-11-16T20:24:25.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3489999999999998e-06 2020-11-16T20:24:25.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675031+0000", 2020-11-16T20:24:25.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060268058999999999 2020-11-16T20:24:25.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735299+0000", 2020-11-16T20:24:25.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7539999999999995e-06 2020-11-16T20:24:25.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735308+0000", 2020-11-16T20:24:25.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7015999999999998e-05 2020-11-16T20:24:25.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735355+0000", 2020-11-16T20:24:25.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017788400000000001 2020-11-16T20:24:25.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735533+0000", 2020-11-16T20:24:25.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060547953000000002 2020-11-16T20:24:25.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:153 3.9s0 3:94376c81:::smithi09216568-11:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.470319537, 2020-11-16T20:24:25.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056909992999999999, 2020-11-16T20:24:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 153 2020-11-16T20:24:25.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:25.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6903999999999998e-05 2020-11-16T20:24:25.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710827+0000", 2020-11-16T20:24:25.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0011e-05 2020-11-16T20:24:25.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710837+0000", 2020-11-16T20:24:25.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4769e-05 2020-11-16T20:24:25.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710851+0000", 2020-11-16T20:24:25.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1129999999999997e-06 2020-11-16T20:24:25.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710858+0000", 2020-11-16T20:24:25.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6598999999999998e-05 2020-11-16T20:24:25.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710914+0000", 2020-11-16T20:24:25.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110771 2020-11-16T20:24:25.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711025+0000", 2020-11-16T20:24:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011061122 2020-11-16T20:24:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.722086+0000", 2020-11-16T20:24:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055101880000000001 2020-11-16T20:24:25.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.727596+0000", 2020-11-16T20:24:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039941002000000003 2020-11-16T20:24:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:25.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767537+0000", 2020-11-16T20:24:25.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000162514 2020-11-16T20:24:25.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767700+0000", 2020-11-16T20:24:25.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056909992999999999 2020-11-16T20:24:25.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.372 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.372 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:154 3.9s0 3:94376c81:::smithi09216568-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.372 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:25.372 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4702000730000009, 2020-11-16T20:24:25.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058295893000000001, 2020-11-16T20:24:25.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:25.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:25.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:25.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2020-11-16T20:24:25.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.374 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:25.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:25.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7756999999999998e-05 2020-11-16T20:24:25.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710947+0000", 2020-11-16T20:24:25.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7429999999999996e-06 2020-11-16T20:24:25.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710954+0000", 2020-11-16T20:24:25.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2998e-05 2020-11-16T20:24:25.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710967+0000", 2020-11-16T20:24:25.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8250000000000004e-06 2020-11-16T20:24:25.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710972+0000", 2020-11-16T20:24:25.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022201499999999999 2020-11-16T20:24:25.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711194+0000", 2020-11-16T20:24:25.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1608e-05 2020-11-16T20:24:25.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2020-11-16T20:24:25.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711225+0000", 2020-11-16T20:24:25.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012795857000000001 2020-11-16T20:24:25.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724021+0000", 2020-11-16T20:24:25.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7220999999999998e-05 2020-11-16T20:24:25.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724098+0000", 2020-11-16T20:24:25.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044522785000000002 2020-11-16T20:24:25.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768621+0000", 2020-11-16T20:24:25.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026034400000000003 2020-11-16T20:24:25.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:25.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768881+0000", 2020-11-16T20:24:25.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032373999999999999 2020-11-16T20:24:25.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.769205+0000", 2020-11-16T20:24:25.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058295893000000001 2020-11-16T20:24:25.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=26, last_complete=31'11, committed=1, applied=1))", 2020-11-16T20:24:25.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:25.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4567603719999997, 2020-11-16T20:24:25.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044143454999999998, 2020-11-16T20:24:25.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:25.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:25.384 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:25.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:25.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:25.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:25.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:25.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:25.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7308999999999999e-05 2020-11-16T20:24:25.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:25.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724376+0000", 2020-11-16T20:24:25.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9070000000000004e-06 2020-11-16T20:24:25.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:25.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724382+0000", 2020-11-16T20:24:25.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042942860999999999 2020-11-16T20:24:25.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:25.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767325+0000", 2020-11-16T20:24:25.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2480000000000001e-05 2020-11-16T20:24:25.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767337+0000", 2020-11-16T20:24:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.177e-05 2020-11-16T20:24:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767399+0000", 2020-11-16T20:24:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010931280000000001 2020-11-16T20:24:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:25.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:25.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:25.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768492+0000", 2020-11-16T20:24:25.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044143454999999998 2020-11-16T20:24:25.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.389 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:25.390 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:25.390 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:25.390 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:25.285+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:25.390 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-11-16T20:24:25.392 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:25.393+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:25.493 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:25.493+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:25.518 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:25.519 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:25.519 INFO:teuthology.orchestra.run.smithi013.stdout: "num_ops": 0 2020-11-16T20:24:25.519 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:25.529 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-11-16T20:24:25.594 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:25.592+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:25.636 INFO:tasks.rados.rados.0.smithi092.stdout:109: done (3 left) 2020-11-16T20:24:25.636 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 28 current snap is 5 2020-11-16T20:24:25.636 INFO:tasks.rados.rados.0.smithi092.stdout:110: seq_num 63 ranges {458752=663552,1122304=401408} 2020-11-16T20:24:25.639 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2020-11-16T20:24:25.643 INFO:tasks.rados.rados.0.smithi092.stdout:110: writing smithi09216568-28 from 458752 to 1122304 tid 1 2020-11-16T20:24:25.645 INFO:tasks.rados.rados.0.smithi092.stdout:110: writing smithi09216568-28 from 1122304 to 1523712 tid 2 2020-11-16T20:24:25.646 INFO:tasks.rados.rados.0.smithi092.stdout:111: read oid 39 snap -1 2020-11-16T20:24:25.646 INFO:tasks.rados.rados.0.smithi092.stdout:111: expect (ObjNum 51 snap 2 seq_num 51) 2020-11-16T20:24:25.646 INFO:tasks.rados.rados.0.smithi092.stdout:112: rollback oid 17 current snap is 5 2020-11-16T20:24:25.646 INFO:tasks.rados.rados.0.smithi092.stdout:112: done (5 left) 2020-11-16T20:24:25.646 INFO:tasks.rados.rados.0.smithi092.stdout:113: copy_from oid 24 from oid 31 current snap is 5 2020-11-16T20:24:25.647 INFO:tasks.rados.rados.0.smithi092.stdout:106: expect (ObjNum 57 snap 4 seq_num 57) 2020-11-16T20:24:25.655 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2020-11-16T20:24:25.659 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:25.659 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:25.659 INFO:teuthology.orchestra.run.smithi013.stdout: "complaint_time": 30, 2020-11-16T20:24:25.659 INFO:teuthology.orchestra.run.smithi013.stdout: "num_blocked_ops": 0 2020-11-16T20:24:25.660 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:25.672 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-11-16T20:24:25.679 INFO:tasks.rados.rados.0.smithi092.stdout:107: finishing write tid 1 to smithi09216568-8 2020-11-16T20:24:25.679 INFO:tasks.rados.rados.0.smithi092.stdout:107: finishing write tid 2 to smithi09216568-8 2020-11-16T20:24:25.679 INFO:tasks.rados.rados.0.smithi092.stdout:107: finishing write tid 3 to smithi09216568-8 2020-11-16T20:24:25.679 INFO:tasks.rados.rados.0.smithi092.stdout:107: finishing write tid 4 to smithi09216568-8 2020-11-16T20:24:25.679 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 8 v 21 (ObjNum 61 snap 5 seq_num 61) dirty exists 2020-11-16T20:24:25.679 INFO:tasks.rados.rados.0.smithi092.stdout:107: left oid 8 (ObjNum 61 snap 5 seq_num 61) 2020-11-16T20:24:25.679 INFO:tasks.rados.rados.0.smithi092.stdout:108: finishing write tid 1 to smithi09216568-50 2020-11-16T20:24:25.680 INFO:tasks.rados.rados.0.smithi092.stdout:108: finishing write tid 2 to smithi09216568-50 2020-11-16T20:24:25.680 INFO:tasks.rados.rados.0.smithi092.stdout:108: finishing write tid 3 to smithi09216568-50 2020-11-16T20:24:25.680 INFO:tasks.rados.rados.0.smithi092.stdout:106: done (5 left) 2020-11-16T20:24:25.680 INFO:tasks.rados.rados.0.smithi092.stdout:107: done (4 left) 2020-11-16T20:24:25.680 INFO:tasks.rados.rados.0.smithi092.stdout:114: read oid 33 snap -1 2020-11-16T20:24:25.680 INFO:tasks.rados.rados.0.smithi092.stdout:114: expect (ObjNum 32 snap 0 seq_num 32) 2020-11-16T20:24:25.680 INFO:tasks.rados.rados.0.smithi092.stdout:115: delete oid 22 current snap is 5 2020-11-16T20:24:25.681 INFO:tasks.rados.rados.0.smithi092.stdout:108: finishing write tid 4 to smithi09216568-50 2020-11-16T20:24:25.681 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 50 v 27 (ObjNum 62 snap 5 seq_num 62) dirty exists 2020-11-16T20:24:25.681 INFO:tasks.rados.rados.0.smithi092.stdout:108: left oid 50 (ObjNum 62 snap 5 seq_num 62) 2020-11-16T20:24:25.681 INFO:tasks.rados.rados.0.smithi092.stdout:111: expect (ObjNum 51 snap 2 seq_num 51) 2020-11-16T20:24:25.717 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:25.716+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:25.730 INFO:tasks.rados.rados.0.smithi092.stdout:110: finishing write tid 1 to smithi09216568-28 2020-11-16T20:24:25.730 INFO:tasks.rados.rados.0.smithi092.stdout:113: finishing copy_from to smithi09216568-24 2020-11-16T20:24:25.731 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 24 v 10 (ObjNum 30 snap 0 seq_num 30) dirty exists 2020-11-16T20:24:25.731 INFO:tasks.rados.rados.0.smithi092.stdout:113: finishing copy_from racing read to smithi09216568-24 2020-11-16T20:24:25.731 INFO:tasks.rados.rados.0.smithi092.stdout:110: finishing write tid 2 to smithi09216568-28 2020-11-16T20:24:25.731 INFO:tasks.rados.rados.0.smithi092.stdout:110: finishing write tid 3 to smithi09216568-28 2020-11-16T20:24:25.731 INFO:tasks.rados.rados.0.smithi092.stdout:110: finishing write tid 4 to smithi09216568-28 2020-11-16T20:24:25.731 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 28 v 21 (ObjNum 63 snap 5 seq_num 63) dirty exists 2020-11-16T20:24:25.731 INFO:tasks.rados.rados.0.smithi092.stdout:110: left oid 28 (ObjNum 63 snap 5 seq_num 63) 2020-11-16T20:24:25.732 INFO:tasks.rados.rados.0.smithi092.stdout:108: done (5 left) 2020-11-16T20:24:25.732 INFO:tasks.rados.rados.0.smithi092.stdout:110: done (4 left) 2020-11-16T20:24:25.732 INFO:tasks.rados.rados.0.smithi092.stdout:111: done (3 left) 2020-11-16T20:24:25.732 INFO:tasks.rados.rados.0.smithi092.stdout:113: done (2 left) 2020-11-16T20:24:25.732 INFO:tasks.rados.rados.0.smithi092.stdout:115: done (1 left) 2020-11-16T20:24:25.733 INFO:tasks.rados.rados.0.smithi092.stdout:116: copy_from oid 43 from oid 34 current snap is 5 2020-11-16T20:24:25.733 INFO:tasks.rados.rados.0.smithi092.stdout:114: expect (ObjNum 32 snap 0 seq_num 32) 2020-11-16T20:24:25.745 INFO:tasks.rados.rados.0.smithi092.stdout:114: done (1 left) 2020-11-16T20:24:25.745 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 49 current snap is 5 2020-11-16T20:24:25.745 INFO:tasks.rados.rados.0.smithi092.stdout:117: seq_num 65 ranges {811008=696320,1507328=401408} 2020-11-16T20:24:25.760 INFO:tasks.rados.rados.0.smithi092.stdout:117: writing smithi09216568-49 from 811008 to 1507328 tid 1 2020-11-16T20:24:25.763 INFO:tasks.rados.rados.0.smithi092.stdout:117: writing smithi09216568-49 from 1507328 to 1908736 tid 2 2020-11-16T20:24:25.763 INFO:tasks.rados.rados.0.smithi092.stdout:118: rollback oid 31 current snap is 5 2020-11-16T20:24:25.763 INFO:tasks.rados.rados.0.smithi092.stdout:118: done (2 left) 2020-11-16T20:24:25.763 INFO:tasks.rados.rados.0.smithi092.stdout:119: setattr oid 33 current snap is 5 2020-11-16T20:24:25.764 INFO:tasks.rados.rados.0.smithi092.stdout:116: finishing copy_from to smithi09216568-43 2020-11-16T20:24:25.764 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 43 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2020-11-16T20:24:25.764 INFO:tasks.rados.rados.0.smithi092.stdout:116: finishing copy_from racing read to smithi09216568-43 2020-11-16T20:24:25.764 INFO:tasks.rados.rados.0.smithi092.stdout:116: done (2 left) 2020-11-16T20:24:25.765 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 24 current snap is 5 2020-11-16T20:24:25.765 INFO:tasks.rados.rados.0.smithi092.stdout:120: seq_num 66 ranges {507904=712704,1220608=401408} 2020-11-16T20:24:25.772 INFO:tasks.rados.rados.0.smithi092.stdout:120: writing smithi09216568-24 from 507904 to 1220608 tid 1 2020-11-16T20:24:25.775 INFO:tasks.rados.rados.0.smithi092.stdout:120: writing smithi09216568-24 from 1220608 to 1622016 tid 2 2020-11-16T20:24:25.775 INFO:tasks.rados.rados.0.smithi092.stdout:121: read oid 28 snap -1 2020-11-16T20:24:25.775 INFO:tasks.rados.rados.0.smithi092.stdout:121: expect (ObjNum 63 snap 5 seq_num 63) 2020-11-16T20:24:25.775 INFO:tasks.rados.rados.0.smithi092.stdout:122: read oid 45 snap -1 2020-11-16T20:24:25.776 INFO:tasks.rados.rados.0.smithi092.stdout:122: expect (ObjNum 44 snap 0 seq_num 44) 2020-11-16T20:24:25.776 INFO:tasks.rados.rados.0.smithi092.stdout:123: rollback oid 6 current snap is 5 2020-11-16T20:24:25.776 INFO:tasks.rados.rados.0.smithi092.stdout:123: done (5 left) 2020-11-16T20:24:25.776 INFO:tasks.rados.rados.0.smithi092.stdout:124: copy_from oid 18 from oid 43 current snap is 5 2020-11-16T20:24:25.776 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 33 v 5 (ObjNum 32 snap 0 seq_num 32) dirty exists 2020-11-16T20:24:25.776 INFO:tasks.rados.rados.0.smithi092.stdout:119: done (5 left) 2020-11-16T20:24:25.776 INFO:tasks.rados.rados.0.smithi092.stdout:125: rollback oid 16 current snap is 5 2020-11-16T20:24:25.776 INFO:tasks.rados.rados.0.smithi092.stdout:125: done (5 left) 2020-11-16T20:24:25.777 INFO:tasks.rados.rados.0.smithi092.stdout:126: rollback oid 36 current snap is 5 2020-11-16T20:24:25.777 INFO:tasks.rados.rados.0.smithi092.stdout:126: done (5 left) 2020-11-16T20:24:25.777 INFO:tasks.rados.rados.0.smithi092.stdout:127: rollback oid 15 current snap is 5 2020-11-16T20:24:25.777 INFO:tasks.rados.rados.0.smithi092.stdout:127: done (5 left) 2020-11-16T20:24:25.777 INFO:tasks.rados.rados.0.smithi092.stdout:128: snap_create 2020-11-16T20:24:25.781 INFO:tasks.rados.rados.0.smithi092.stdout:122: expect (ObjNum 44 snap 0 seq_num 44) 2020-11-16T20:24:25.796 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:25.797+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:25.806 INFO:tasks.rados.rados.0.smithi092.stdout:117: finishing write tid 1 to smithi09216568-49 2020-11-16T20:24:25.806 INFO:tasks.rados.rados.0.smithi092.stdout:117: finishing write tid 2 to smithi09216568-49 2020-11-16T20:24:25.807 INFO:tasks.rados.rados.0.smithi092.stdout:121: expect (ObjNum 63 snap 5 seq_num 63) 2020-11-16T20:24:25.858 INFO:tasks.rados.rados.0.smithi092.stdout:120: finishing write tid 1 to smithi09216568-24 2020-11-16T20:24:25.858 INFO:tasks.rados.rados.0.smithi092.stdout:120: finishing write tid 2 to smithi09216568-24 2020-11-16T20:24:25.859 INFO:tasks.rados.rados.0.smithi092.stdout:120: finishing write tid 3 to smithi09216568-24 2020-11-16T20:24:25.859 INFO:tasks.rados.rados.0.smithi092.stdout:120: finishing write tid 4 to smithi09216568-24 2020-11-16T20:24:25.859 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 24 v 31 (ObjNum 66 snap 5 seq_num 66) dirty exists 2020-11-16T20:24:25.859 INFO:tasks.rados.rados.0.smithi092.stdout:120: left oid 24 (ObjNum 66 snap 5 seq_num 66) 2020-11-16T20:24:25.859 INFO:tasks.rados.rados.0.smithi092.stdout:124: finishing copy_from to smithi09216568-18 2020-11-16T20:24:25.859 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 18 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2020-11-16T20:24:25.859 INFO:tasks.rados.rados.0.smithi092.stdout:124: finishing copy_from racing read to smithi09216568-18 2020-11-16T20:24:25.860 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:25.861 INFO:teuthology.orchestra.run.smithi013.stdout: "size": 20, 2020-11-16T20:24:25.861 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 600, 2020-11-16T20:24:25.861 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [ 2020-11-16T20:24:25.861 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.861 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:56 3.8s0 3:159ef6eb:::smithi09216568-19:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.861 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.570140+0000", 2020-11-16T20:24:25.862 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.287816039999999, 2020-11-16T20:24:25.862 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028987748000000001, 2020-11-16T20:24:25.862 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.862 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.862 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.862 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.862 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.863 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 56 2020-11-16T20:24:25.863 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.863 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.863 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.863 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.863 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570140+0000", 2020-11-16T20:24:25.863 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.863 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.864 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.864 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.864 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570140+0000", 2020-11-16T20:24:25.864 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.9674000000000001e-05 2020-11-16T20:24:25.864 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.864 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.864 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.865 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570169+0000", 2020-11-16T20:24:25.865 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.1849999999999997e-06 2020-11-16T20:24:25.865 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.865 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.865 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.865 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570173+0000", 2020-11-16T20:24:25.865 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.7610000000000002e-06 2020-11-16T20:24:25.865 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.866 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.866 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.866 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570180+0000", 2020-11-16T20:24:25.866 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.5719999999999996e-06 2020-11-16T20:24:25.866 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.866 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.866 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.867 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570185+0000", 2020-11-16T20:24:25.867 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0046248579999999999 2020-11-16T20:24:25.867 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.867 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.867 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.867 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.574810+0000", 2020-11-16T20:24:25.867 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000105401 2020-11-16T20:24:25.867 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.868 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.868 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.868 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.574915+0000", 2020-11-16T20:24:25.868 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0077109320000000002 2020-11-16T20:24:25.868 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.868 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.868 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.869 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.582626+0000", 2020-11-16T20:24:25.869 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.5486e-05 2020-11-16T20:24:25.869 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.869 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.869 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.869 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.582722+0000", 2020-11-16T20:24:25.869 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00088386900000000002 2020-11-16T20:24:25.869 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.870 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.870 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.870 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.583605+0000", 2020-11-16T20:24:25.870 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00125154 2020-11-16T20:24:25.870 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.870 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.870 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.871 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.584857+0000", 2020-11-16T20:24:25.871 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.01416066 2020-11-16T20:24:25.871 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.871 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.871 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:25.871 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599018+0000", 2020-11-16T20:24:25.871 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00010980999999999999 2020-11-16T20:24:25.871 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.872 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.872 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.872 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599127+0000", 2020-11-16T20:24:25.872 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028987748000000001 2020-11-16T20:24:25.872 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.872 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.872 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.873 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.873 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.873 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:57 3.8s0 3:159ef6eb:::smithi09216568-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.873 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.570222+0000", 2020-11-16T20:24:25.873 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.287734025000001, 2020-11-16T20:24:25.873 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.041859702999999998, 2020-11-16T20:24:25.873 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.873 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:25.874 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.874 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.874 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.874 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 57 2020-11-16T20:24:25.874 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.874 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.874 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.875 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.875 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570222+0000", 2020-11-16T20:24:25.875 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.875 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.875 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.875 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.875 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570222+0000", 2020-11-16T20:24:25.875 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.0971999999999998e-05 2020-11-16T20:24:25.876 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.876 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.876 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.876 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570243+0000", 2020-11-16T20:24:25.876 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.301e-06 2020-11-16T20:24:25.876 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.876 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.877 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.877 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570247+0000", 2020-11-16T20:24:25.877 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.5560000000000002e-06 2020-11-16T20:24:25.877 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.877 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.877 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.877 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570254+0000", 2020-11-16T20:24:25.877 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.2610000000000002e-06 2020-11-16T20:24:25.878 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.878 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.878 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.878 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570258+0000", 2020-11-16T20:24:25.878 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0053828970000000002 2020-11-16T20:24:25.878 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.878 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.879 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.879 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.575641+0000", 2020-11-16T20:24:25.879 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000103824 2020-11-16T20:24:25.879 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.879 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.879 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.879 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.575744+0000", 2020-11-16T20:24:25.879 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.008211444 2020-11-16T20:24:25.880 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.880 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.880 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.880 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.583956+0000", 2020-11-16T20:24:25.880 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000105116 2020-11-16T20:24:25.880 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.880 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.880 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.881 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.584061+0000", 2020-11-16T20:24:25.881 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.015111940000000001 2020-11-16T20:24:25.881 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.881 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.881 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.881 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599173+0000", 2020-11-16T20:24:25.881 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.3417999999999995e-05 2020-11-16T20:24:25.882 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.882 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.882 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.882 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599236+0000", 2020-11-16T20:24:25.882 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012844974 2020-11-16T20:24:25.882 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.882 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.882 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.883 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612081+0000", 2020-11-16T20:24:25.883 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.041859702999999998 2020-11-16T20:24:25.883 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.883 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.883 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.883 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.883 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.884 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:74 3.8s0 3:1ae67b5f:::smithi09216568-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.884 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.587001+0000", 2020-11-16T20:24:25.884 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.270954744999999, 2020-11-16T20:24:25.884 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.048715451999999999, 2020-11-16T20:24:25.884 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.884 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.884 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.884 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.885 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.885 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 74 2020-11-16T20:24:25.885 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.885 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.885 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.885 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.885 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587001+0000", 2020-11-16T20:24:25.886 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.886 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.886 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.886 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.886 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587001+0000", 2020-11-16T20:24:25.886 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.4703999999999997e-05 2020-11-16T20:24:25.886 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.886 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.887 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.887 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587056+0000", 2020-11-16T20:24:25.887 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.6519999999999995e-06 2020-11-16T20:24:25.887 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.887 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.887 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.887 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587064+0000", 2020-11-16T20:24:25.888 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.0737000000000001e-05 2020-11-16T20:24:25.888 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.888 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.888 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.888 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587075+0000", 2020-11-16T20:24:25.888 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.0350000000000003e-06 2020-11-16T20:24:25.888 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.888 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.889 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.889 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587079+0000", 2020-11-16T20:24:25.889 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0062840250000000004 2020-11-16T20:24:25.889 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.889 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.889 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.889 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.593363+0000", 2020-11-16T20:24:25.890 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000125931 2020-11-16T20:24:25.890 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.890 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.890 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.890 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.593489+0000", 2020-11-16T20:24:25.890 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017552346999999999 2020-11-16T20:24:25.890 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.890 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.891 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.891 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611041+0000", 2020-11-16T20:24:25.891 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000105302 2020-11-16T20:24:25.891 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.891 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.891 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.891 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611147+0000", 2020-11-16T20:24:25.892 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00032716199999999998 2020-11-16T20:24:25.892 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.892 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.892 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.892 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611474+0000", 2020-11-16T20:24:25.892 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0036760360000000001 2020-11-16T20:24:25.892 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.892 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.893 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.893 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.615150+0000", 2020-11-16T20:24:25.893 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.020462314999999998 2020-11-16T20:24:25.893 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.893 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.893 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:25.893 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635612+0000", 2020-11-16T20:24:25.894 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000104206 2020-11-16T20:24:25.894 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.894 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.894 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.894 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635716+0000", 2020-11-16T20:24:25.894 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.048715451999999999 2020-11-16T20:24:25.894 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.894 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.895 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.895 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.895 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.895 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:75 3.8s0 3:1ae67b5f:::smithi09216568-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.895 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.587131+0000", 2020-11-16T20:24:25.895 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.270825059, 2020-11-16T20:24:25.895 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.050649959000000001, 2020-11-16T20:24:25.895 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.896 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:25.896 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.896 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.896 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.896 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 75 2020-11-16T20:24:25.896 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.896 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.897 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.897 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.897 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587131+0000", 2020-11-16T20:24:25.897 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.897 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.897 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.897 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.897 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587131+0000", 2020-11-16T20:24:25.898 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.2969000000000001e-05 2020-11-16T20:24:25.898 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.898 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.898 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.898 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587174+0000", 2020-11-16T20:24:25.898 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.9169999999999995e-06 2020-11-16T20:24:25.898 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.899 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.899 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.899 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587183+0000", 2020-11-16T20:24:25.899 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4047e-05 2020-11-16T20:24:25.899 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.899 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.899 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.899 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587197+0000", 2020-11-16T20:24:25.900 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.2350000000000002e-06 2020-11-16T20:24:25.900 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.900 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.900 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.900 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587204+0000", 2020-11-16T20:24:25.900 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0064723080000000004 2020-11-16T20:24:25.900 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.900 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.901 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.901 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.593676+0000", 2020-11-16T20:24:25.901 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00014109800000000001 2020-11-16T20:24:25.901 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.901 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.901 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.901 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.593817+0000", 2020-11-16T20:24:25.902 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017874455000000001 2020-11-16T20:24:25.902 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.902 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.902 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.902 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611692+0000", 2020-11-16T20:24:25.902 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.5184000000000002e-05 2020-11-16T20:24:25.902 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.902 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.903 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.903 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611787+0000", 2020-11-16T20:24:25.903 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.024003231999999999 2020-11-16T20:24:25.903 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.903 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.903 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.903 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635790+0000", 2020-11-16T20:24:25.904 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000107527 2020-11-16T20:24:25.904 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.904 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.904 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.904 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635898+0000", 2020-11-16T20:24:25.904 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001882987 2020-11-16T20:24:25.904 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.904 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.905 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.905 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.637781+0000", 2020-11-16T20:24:25.905 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.050649959000000001 2020-11-16T20:24:25.905 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.905 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.905 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.905 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.905 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.906 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:80 3.6s0 3:6ded7fc6:::smithi09216568-27:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.906 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.598238+0000", 2020-11-16T20:24:25.906 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.259717514, 2020-11-16T20:24:25.906 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.035012275000000002, 2020-11-16T20:24:25.906 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.906 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.906 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.907 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.907 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.907 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 80 2020-11-16T20:24:25.907 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.907 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.907 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.907 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.908 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598238+0000", 2020-11-16T20:24:25.908 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.908 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.908 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.908 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.908 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598238+0000", 2020-11-16T20:24:25.908 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.9252000000000001e-05 2020-11-16T20:24:25.908 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.909 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.909 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.909 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598297+0000", 2020-11-16T20:24:25.909 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.9910000000000002e-06 2020-11-16T20:24:25.909 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.909 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.909 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.909 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598305+0000", 2020-11-16T20:24:25.910 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4090999999999999e-05 2020-11-16T20:24:25.910 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.910 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.910 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.910 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598319+0000", 2020-11-16T20:24:25.910 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.2589999999999997e-06 2020-11-16T20:24:25.910 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.911 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.911 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.911 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598326+0000", 2020-11-16T20:24:25.911 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.006674915 2020-11-16T20:24:25.911 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.911 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.911 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.912 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.605001+0000", 2020-11-16T20:24:25.912 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000152025 2020-11-16T20:24:25.912 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.912 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.912 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.912 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.605153+0000", 2020-11-16T20:24:25.912 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013891341 2020-11-16T20:24:25.912 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.913 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.913 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.913 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619044+0000", 2020-11-16T20:24:25.913 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00010469 2020-11-16T20:24:25.913 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.913 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.913 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.914 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619149+0000", 2020-11-16T20:24:25.914 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00041109499999999999 2020-11-16T20:24:25.914 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.914 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.914 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.914 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619560+0000", 2020-11-16T20:24:25.914 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00083761899999999995 2020-11-16T20:24:25.914 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.915 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.915 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.915 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.620397+0000", 2020-11-16T20:24:25.915 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012771629 2020-11-16T20:24:25.915 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.915 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.915 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:25.916 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.633169+0000", 2020-11-16T20:24:25.916 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.1367999999999998e-05 2020-11-16T20:24:25.916 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.916 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.916 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.916 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.633250+0000", 2020-11-16T20:24:25.916 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.035012275000000002 2020-11-16T20:24:25.916 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.917 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.917 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.917 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.917 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.917 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:81 3.6s0 3:6ded7fc6:::smithi09216568-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.917 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.598374+0000", 2020-11-16T20:24:25.917 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.259582062, 2020-11-16T20:24:25.918 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036153288999999998, 2020-11-16T20:24:25.918 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.918 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:25.918 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.918 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.918 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.918 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 81 2020-11-16T20:24:25.918 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.919 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.919 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.919 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.919 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598374+0000", 2020-11-16T20:24:25.919 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.919 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.919 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.920 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.920 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598374+0000", 2020-11-16T20:24:25.920 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.3807000000000001e-05 2020-11-16T20:24:25.920 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.920 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.920 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.920 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598397+0000", 2020-11-16T20:24:25.920 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.3869999999999997e-06 2020-11-16T20:24:25.921 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.921 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.921 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.921 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598402+0000", 2020-11-16T20:24:25.921 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.2539999999999992e-06 2020-11-16T20:24:25.921 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.921 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.922 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.922 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598410+0000", 2020-11-16T20:24:25.922 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.5009999999999998e-06 2020-11-16T20:24:25.922 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.922 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.922 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.922 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598415+0000", 2020-11-16T20:24:25.922 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0067833110000000002 2020-11-16T20:24:25.923 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.923 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.923 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.923 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.605198+0000", 2020-11-16T20:24:25.923 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0051836130000000001 2020-11-16T20:24:25.923 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.923 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.923 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.924 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.610381+0000", 2020-11-16T20:24:25.924 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0093756299999999994 2020-11-16T20:24:25.924 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.924 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.924 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.924 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619757+0000", 2020-11-16T20:24:25.924 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.0941999999999998e-05 2020-11-16T20:24:25.925 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.925 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.925 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.925 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619848+0000", 2020-11-16T20:24:25.925 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013541793999999999 2020-11-16T20:24:25.925 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.925 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.925 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.926 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.633390+0000", 2020-11-16T20:24:25.926 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00010958700000000001 2020-11-16T20:24:25.926 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.926 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.926 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.926 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.633499+0000", 2020-11-16T20:24:25.926 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0010274629999999999 2020-11-16T20:24:25.927 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.927 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.927 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.927 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.634527+0000", 2020-11-16T20:24:25.927 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036153288999999998 2020-11-16T20:24:25.927 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.927 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.927 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.928 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.928 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.928 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:82 3.8s0 3:18af2eec:::smithi09216568-28:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.928 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.598442+0000", 2020-11-16T20:24:25.928 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.259513394000001, 2020-11-16T20:24:25.928 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030589345, 2020-11-16T20:24:25.928 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.928 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.929 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.929 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.929 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.929 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 82 2020-11-16T20:24:25.929 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.929 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.929 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.930 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.930 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598442+0000", 2020-11-16T20:24:25.930 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.930 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.930 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.930 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.930 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598442+0000", 2020-11-16T20:24:25.930 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00048240000000000002 2020-11-16T20:24:25.931 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.931 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.931 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.931 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598925+0000", 2020-11-16T20:24:25.931 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.8969999999999996e-06 2020-11-16T20:24:25.931 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.931 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.932 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.932 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598931+0000", 2020-11-16T20:24:25.932 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.8952000000000001e-05 2020-11-16T20:24:25.932 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.932 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.932 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.932 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598950+0000", 2020-11-16T20:24:25.932 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.9427000000000001e-05 2020-11-16T20:24:25.933 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.933 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.933 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.933 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598979+0000", 2020-11-16T20:24:25.933 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00048842999999999998 2020-11-16T20:24:25.933 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.933 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.934 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.934 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599467+0000", 2020-11-16T20:24:25.934 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000186918 2020-11-16T20:24:25.934 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.934 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.934 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.934 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599654+0000", 2020-11-16T20:24:25.934 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0058116189999999996 2020-11-16T20:24:25.935 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.935 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.935 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.935 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.605466+0000", 2020-11-16T20:24:25.935 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0094244389999999997 2020-11-16T20:24:25.935 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.935 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.936 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.936 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.614890+0000", 2020-11-16T20:24:25.936 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013836742000000001 2020-11-16T20:24:25.936 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.936 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.936 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:25.936 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.628727+0000", 2020-11-16T20:24:25.936 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00030452099999999998 2020-11-16T20:24:25.937 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.937 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.937 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.937 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.629032+0000", 2020-11-16T20:24:25.937 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030589345 2020-11-16T20:24:25.937 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.937 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.937 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.938 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.938 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.938 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:83 3.8s0 3:18af2eec:::smithi09216568-28:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.938 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.599013+0000", 2020-11-16T20:24:25.938 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.25894227, 2020-11-16T20:24:25.938 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.037367409999999997, 2020-11-16T20:24:25.938 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.939 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.939 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.939 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.939 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.939 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 83 2020-11-16T20:24:25.939 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.939 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.939 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.940 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.940 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599013+0000", 2020-11-16T20:24:25.940 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.940 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.940 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.940 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.940 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599013+0000", 2020-11-16T20:24:25.941 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.1021e-05 2020-11-16T20:24:25.941 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.941 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.941 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.941 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599044+0000", 2020-11-16T20:24:25.941 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.2189999999999998e-06 2020-11-16T20:24:25.941 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.941 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.942 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.942 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599049+0000", 2020-11-16T20:24:25.942 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.238e-06 2020-11-16T20:24:25.942 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.942 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.942 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.942 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599056+0000", 2020-11-16T20:24:25.943 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.2409999999999997e-06 2020-11-16T20:24:25.943 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.943 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.943 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.943 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599060+0000", 2020-11-16T20:24:25.943 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013073023 2020-11-16T20:24:25.943 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.943 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.944 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.944 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612133+0000", 2020-11-16T20:24:25.944 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00010226999999999999 2020-11-16T20:24:25.944 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.944 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.944 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.944 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612235+0000", 2020-11-16T20:24:25.944 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.016896411 2020-11-16T20:24:25.945 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.945 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.945 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.945 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.629132+0000", 2020-11-16T20:24:25.945 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.9254999999999998e-05 2020-11-16T20:24:25.945 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.945 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.946 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.946 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.629231+0000", 2020-11-16T20:24:25.946 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00051924100000000004 2020-11-16T20:24:25.946 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.946 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.946 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.946 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.629750+0000", 2020-11-16T20:24:25.946 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001168388 2020-11-16T20:24:25.947 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.947 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.947 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.947 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.630919+0000", 2020-11-16T20:24:25.947 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0053587779999999998 2020-11-16T20:24:25.947 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.947 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.948 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:25.948 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.636277+0000", 2020-11-16T20:24:25.948 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000103325 2020-11-16T20:24:25.948 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.948 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.948 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.948 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.636381+0000", 2020-11-16T20:24:25.948 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.037367409999999997 2020-11-16T20:24:25.949 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.949 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.949 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.949 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.949 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.949 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:84 3.8s0 3:18af2eec:::smithi09216568-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.949 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.599104+0000", 2020-11-16T20:24:25.949 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.258851610000001, 2020-11-16T20:24:25.950 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.038956530000000003, 2020-11-16T20:24:25.950 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.950 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:25.950 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.950 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.950 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.950 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 84 2020-11-16T20:24:25.951 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.951 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.951 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.951 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.951 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599104+0000", 2020-11-16T20:24:25.951 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.951 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.951 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.952 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.952 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599104+0000", 2020-11-16T20:24:25.952 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.8121000000000001e-05 2020-11-16T20:24:25.952 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.952 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.952 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.952 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599132+0000", 2020-11-16T20:24:25.952 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.9519999999999996e-06 2020-11-16T20:24:25.953 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.953 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.953 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.953 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599136+0000", 2020-11-16T20:24:25.953 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.6709999999999999e-06 2020-11-16T20:24:25.953 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.953 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.954 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.954 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599144+0000", 2020-11-16T20:24:25.954 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.5540000000000002e-06 2020-11-16T20:24:25.954 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.954 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.954 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.954 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599147+0000", 2020-11-16T20:24:25.954 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013124421000000001 2020-11-16T20:24:25.955 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.955 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.955 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.955 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612272+0000", 2020-11-16T20:24:25.955 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.2768000000000006e-05 2020-11-16T20:24:25.955 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.955 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.955 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.956 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612335+0000", 2020-11-16T20:24:25.956 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017973705 2020-11-16T20:24:25.956 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.956 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.956 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.956 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.630308+0000", 2020-11-16T20:24:25.956 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.1384999999999994e-05 2020-11-16T20:24:25.957 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.957 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.957 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.957 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.630390+0000", 2020-11-16T20:24:25.957 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0063989729999999996 2020-11-16T20:24:25.957 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.957 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.957 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.958 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.636789+0000", 2020-11-16T20:24:25.958 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00013566300000000001 2020-11-16T20:24:25.958 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.958 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.958 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.958 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.636924+0000", 2020-11-16T20:24:25.958 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001136317 2020-11-16T20:24:25.959 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.959 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.959 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.959 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.638061+0000", 2020-11-16T20:24:25.959 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.038956530000000003 2020-11-16T20:24:25.959 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.959 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.959 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.960 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.960 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.960 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:93 3.6s0 3:6a9c2d76:::smithi09216568-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.960 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.608124+0000", 2020-11-16T20:24:25.960 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.249831767, 2020-11-16T20:24:25.960 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028905492000000001, 2020-11-16T20:24:25.960 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.960 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:25.961 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.961 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.961 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.961 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 93 2020-11-16T20:24:25.961 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.961 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.961 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.962 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.962 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.608124+0000", 2020-11-16T20:24:25.962 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.962 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.962 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.962 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.962 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.608124+0000", 2020-11-16T20:24:25.962 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.3887000000000002e-05 2020-11-16T20:24:25.963 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.963 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.963 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.963 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.608168+0000", 2020-11-16T20:24:25.963 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.3220000000000004e-06 2020-11-16T20:24:25.963 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.963 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.964 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.964 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.608175+0000", 2020-11-16T20:24:25.964 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4198e-05 2020-11-16T20:24:25.964 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.964 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.964 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.964 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.608189+0000", 2020-11-16T20:24:25.964 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.5559999999999998e-06 2020-11-16T20:24:25.965 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.965 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.965 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.965 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.608195+0000", 2020-11-16T20:24:25.965 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.011690717 2020-11-16T20:24:25.965 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.965 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.966 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.966 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619886+0000", 2020-11-16T20:24:25.966 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.0007999999999996e-05 2020-11-16T20:24:25.966 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.966 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.966 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.966 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619976+0000", 2020-11-16T20:24:25.966 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012670712000000001 2020-11-16T20:24:25.967 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.967 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.967 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.967 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.632646+0000", 2020-11-16T20:24:25.967 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000115999 2020-11-16T20:24:25.967 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.967 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.967 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.968 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.632762+0000", 2020-11-16T20:24:25.968 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0023891250000000002 2020-11-16T20:24:25.968 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.968 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.968 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.968 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635151+0000", 2020-11-16T20:24:25.968 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000112803 2020-11-16T20:24:25.969 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.969 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.969 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.969 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635264+0000", 2020-11-16T20:24:25.969 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001765165 2020-11-16T20:24:25.969 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.969 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.969 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.970 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.637029+0000", 2020-11-16T20:24:25.970 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028905492000000001 2020-11-16T20:24:25.970 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.970 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.970 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.970 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.970 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.971 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:128 3.2s0 3:414e2876:::smithi09216568-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.971 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.671586+0000", 2020-11-16T20:24:25.971 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.186369751999999, 2020-11-16T20:24:25.971 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028910251000000001, 2020-11-16T20:24:25.971 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.971 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.971 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.971 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.972 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.972 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 128 2020-11-16T20:24:25.972 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.972 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.972 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.972 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.972 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671586+0000", 2020-11-16T20:24:25.972 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.973 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.973 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.973 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.973 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671586+0000", 2020-11-16T20:24:25.973 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.3176000000000006e-05 2020-11-16T20:24:25.973 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.973 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.974 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.974 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671649+0000", 2020-11-16T20:24:25.974 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.8970000000000002e-06 2020-11-16T20:24:25.974 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.974 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.974 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.974 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671656+0000", 2020-11-16T20:24:25.974 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.3686000000000001e-05 2020-11-16T20:24:25.975 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.975 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.975 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.975 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671670+0000", 2020-11-16T20:24:25.975 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.9460000000000003e-06 2020-11-16T20:24:25.975 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.975 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.975 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.976 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671676+0000", 2020-11-16T20:24:25.976 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.006470857 2020-11-16T20:24:25.976 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.976 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.976 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.976 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.678146+0000", 2020-11-16T20:24:25.976 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.6054e-05 2020-11-16T20:24:25.977 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.977 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.977 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.977 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.678233+0000", 2020-11-16T20:24:25.977 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.016952812000000001 2020-11-16T20:24:25.977 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.977 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.977 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.978 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.695185+0000", 2020-11-16T20:24:25.978 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00012091699999999999 2020-11-16T20:24:25.978 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.978 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.978 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.978 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.695306+0000", 2020-11-16T20:24:25.978 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00058599499999999996 2020-11-16T20:24:25.979 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.979 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.979 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:25.979 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.695892+0000", 2020-11-16T20:24:25.979 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00084612299999999995 2020-11-16T20:24:25.979 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.979 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.979 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:25.980 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.696738+0000", 2020-11-16T20:24:25.980 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.003639818 2020-11-16T20:24:25.980 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.980 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.980 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:25.980 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700378+0000", 2020-11-16T20:24:25.980 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00011797 2020-11-16T20:24:25.980 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.981 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.981 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.981 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700496+0000", 2020-11-16T20:24:25.981 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028910251000000001 2020-11-16T20:24:25.981 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.981 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.981 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.981 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.982 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.982 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:129 3.2s0 3:414e2876:::smithi09216568-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:25.982 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.671721+0000", 2020-11-16T20:24:25.982 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.186234962, 2020-11-16T20:24:25.982 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.041126508999999999, 2020-11-16T20:24:25.982 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.982 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:25.983 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.983 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.983 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.983 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 129 2020-11-16T20:24:25.983 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.983 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.983 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.983 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.984 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671721+0000", 2020-11-16T20:24:25.984 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.984 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.984 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.984 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.984 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671721+0000", 2020-11-16T20:24:25.984 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.1564999999999999e-05 2020-11-16T20:24:25.985 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.985 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.985 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.985 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671762+0000", 2020-11-16T20:24:25.985 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.2339999999999997e-06 2020-11-16T20:24:25.985 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.985 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.985 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.986 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671770+0000", 2020-11-16T20:24:25.986 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.501e-05 2020-11-16T20:24:25.986 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.986 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.986 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.986 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671785+0000", 2020-11-16T20:24:25.986 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.6481e-05 2020-11-16T20:24:25.986 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.987 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.987 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.987 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671801+0000", 2020-11-16T20:24:25.987 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0064728160000000002 2020-11-16T20:24:25.987 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.987 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.987 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.988 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.678274+0000", 2020-11-16T20:24:25.988 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.0265e-05 2020-11-16T20:24:25.988 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.988 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.988 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.988 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.678334+0000", 2020-11-16T20:24:25.988 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017875683 2020-11-16T20:24:25.988 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.989 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.989 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.989 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.696210+0000", 2020-11-16T20:24:25.989 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00011280700000000001 2020-11-16T20:24:25.989 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.989 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.989 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.990 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.696323+0000", 2020-11-16T20:24:25.990 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0042602059999999999 2020-11-16T20:24:25.990 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.990 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.990 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.990 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700583+0000", 2020-11-16T20:24:25.990 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.6906999999999999e-05 2020-11-16T20:24:25.990 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.991 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.991 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:25.991 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700680+0000", 2020-11-16T20:24:25.991 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012167535 2020-11-16T20:24:25.991 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.991 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.991 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:25.992 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.712847+0000", 2020-11-16T20:24:25.992 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.041126508999999999 2020-11-16T20:24:25.992 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.992 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:25.992 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:25.992 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.992 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.992 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:131 3.2s0 3:47d6be26:::smithi09216568-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:25.993 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.674850+0000", 2020-11-16T20:24:25.993 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.183105386999999, 2020-11-16T20:24:25.993 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036389118999999998, 2020-11-16T20:24:25.993 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:25.993 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:25.993 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:25.993 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:25.993 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:25.994 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 131 2020-11-16T20:24:25.994 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.994 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:25.994 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.994 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:25.994 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674850+0000", 2020-11-16T20:24:25.994 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:25.995 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.995 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.995 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:25.995 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674850+0000", 2020-11-16T20:24:25.995 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.0031000000000001e-05 2020-11-16T20:24:25.995 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.995 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.996 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:25.996 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674880+0000", 2020-11-16T20:24:25.996 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.6959999999999998e-06 2020-11-16T20:24:25.996 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.996 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.996 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:25.996 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674884+0000", 2020-11-16T20:24:25.996 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.9369999999999998e-06 2020-11-16T20:24:25.997 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.997 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.997 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:25.997 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674891+0000", 2020-11-16T20:24:25.997 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.7549999999999999e-06 2020-11-16T20:24:25.997 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.997 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.997 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:25.998 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674894+0000", 2020-11-16T20:24:25.998 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012029560999999999 2020-11-16T20:24:25.998 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.998 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.998 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:25.998 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.686923+0000", 2020-11-16T20:24:25.998 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.6857e-05 2020-11-16T20:24:25.999 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.999 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.999 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:25.999 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.686990+0000", 2020-11-16T20:24:25.999 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.011909622999999999 2020-11-16T20:24:25.999 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:25.999 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:25.999 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.000 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.698900+0000", 2020-11-16T20:24:26.000 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000178133 2020-11-16T20:24:26.000 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.000 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.000 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:26.000 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.699078+0000", 2020-11-16T20:24:26.000 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00066865499999999999 2020-11-16T20:24:26.000 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.001 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.001 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:26.001 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.699747+0000", 2020-11-16T20:24:26.001 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0013938620000000001 2020-11-16T20:24:26.001 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.001 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.001 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:26.002 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.701140+0000", 2020-11-16T20:24:26.002 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0060941889999999999 2020-11-16T20:24:26.002 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.002 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.002 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:26.002 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.707235+0000", 2020-11-16T20:24:26.002 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0040048200000000001 2020-11-16T20:24:26.002 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.003 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.003 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:26.003 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.711239+0000", 2020-11-16T20:24:26.003 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036389118999999998 2020-11-16T20:24:26.003 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.003 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.003 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.004 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.004 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.004 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:132 3.2s0 3:47d6be26:::smithi09216568-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:26.004 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.674919+0000", 2020-11-16T20:24:26.004 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.183036498, 2020-11-16T20:24:26.004 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036684337999999997, 2020-11-16T20:24:26.004 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:26.004 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "reached pg", 2020-11-16T20:24:26.005 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:26.005 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:26.005 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:26.005 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 132 2020-11-16T20:24:26.005 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.005 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:26.005 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.005 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:26.006 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674919+0000", 2020-11-16T20:24:26.006 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:26.006 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.006 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.006 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:26.006 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674919+0000", 2020-11-16T20:24:26.006 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.0027e-05 2020-11-16T20:24:26.007 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.007 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.007 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:26.007 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674939+0000", 2020-11-16T20:24:26.007 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.6299999999999997e-06 2020-11-16T20:24:26.007 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.007 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.007 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:26.008 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674944+0000", 2020-11-16T20:24:26.008 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.8390000000000001e-06 2020-11-16T20:24:26.008 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.008 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.008 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:26.008 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674951+0000", 2020-11-16T20:24:26.008 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.7250000000000002e-06 2020-11-16T20:24:26.009 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.009 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.009 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:26.009 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674953+0000", 2020-11-16T20:24:26.009 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012060743000000001 2020-11-16T20:24:26.009 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.009 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.009 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.010 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.687014+0000", 2020-11-16T20:24:26.010 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.0745000000000002e-05 2020-11-16T20:24:26.010 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.010 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.010 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:26.010 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.687075+0000", 2020-11-16T20:24:26.010 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013032431000000001 2020-11-16T20:24:26.010 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.011 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.011 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.011 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700107+0000", 2020-11-16T20:24:26.011 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00011721 2020-11-16T20:24:26.011 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.011 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.011 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:26.012 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700225+0000", 2020-11-16T20:24:26.012 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.011367998000000001 2020-11-16T20:24:26.012 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.012 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.012 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.012 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.711593+0000", 2020-11-16T20:24:26.012 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.099e-05 2020-11-16T20:24:26.012 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.013 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.013 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:26.013 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.711604+0000", 2020-11-16T20:24:26.013 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036684337999999997 2020-11-16T20:24:26.013 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.013 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.013 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.013 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.014 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.014 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:132 3.2s0 3:47d6be26:::smithi09216568-44:head [read 0~1] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:26.014 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.715261+0000", 2020-11-16T20:24:26.014 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.142694326000001, 2020-11-16T20:24:26.014 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028833008, 2020-11-16T20:24:26.014 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:26.014 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:26.015 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:26.015 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:26.015 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:26.015 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 132 2020-11-16T20:24:26.015 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.015 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:26.015 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.015 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:26.016 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.715261+0000", 2020-11-16T20:24:26.016 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:26.016 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.016 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.016 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:26.016 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.715261+0000", 2020-11-16T20:24:26.016 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.1294999999999997e-05 2020-11-16T20:24:26.017 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.017 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.017 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:26.017 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.715303+0000", 2020-11-16T20:24:26.017 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.9450000000000004e-06 2020-11-16T20:24:26.017 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.017 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.017 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:26.018 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.715310+0000", 2020-11-16T20:24:26.018 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.3023999999999999e-05 2020-11-16T20:24:26.018 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.018 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.018 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:26.018 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.715323+0000", 2020-11-16T20:24:26.018 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.4470000000000003e-06 2020-11-16T20:24:26.019 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.019 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.019 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:26.019 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.715329+0000", 2020-11-16T20:24:26.019 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00015361500000000001 2020-11-16T20:24:26.019 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.019 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.019 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.020 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.715483+0000", 2020-11-16T20:24:26.020 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.2884000000000001e-05 2020-11-16T20:24:26.020 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.020 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.020 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:26.020 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.715566+0000", 2020-11-16T20:24:26.020 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028528798000000001 2020-11-16T20:24:26.020 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.021 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.021 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:26.021 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.744094+0000", 2020-11-16T20:24:26.021 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028833008 2020-11-16T20:24:26.021 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.021 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.021 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.022 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.022 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.022 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:152 3.7s0 3:eba7105b:::smithi09216568-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:26.022 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.719485+0000", 2020-11-16T20:24:26.022 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 10.138470691, 2020-11-16T20:24:26.022 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.38311783900000002, 2020-11-16T20:24:26.022 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:26.022 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:26.023 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:26.023 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:26.023 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:26.023 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 152 2020-11-16T20:24:26.023 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.023 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:26.023 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.024 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:26.024 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719485+0000", 2020-11-16T20:24:26.024 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:26.024 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.024 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.024 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:26.024 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719485+0000", 2020-11-16T20:24:26.024 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.2246999999999999e-05 2020-11-16T20:24:26.025 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.025 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.025 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:26.025 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719507+0000", 2020-11-16T20:24:26.025 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.5299999999999998e-06 2020-11-16T20:24:26.025 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.025 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.025 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:26.026 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719512+0000", 2020-11-16T20:24:26.026 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.0484e-05 2020-11-16T20:24:26.026 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.026 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.026 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:26.026 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719522+0000", 2020-11-16T20:24:26.026 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.783e-06 2020-11-16T20:24:26.027 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.027 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.027 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:26.027 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719526+0000", 2020-11-16T20:24:26.027 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.007154166 2020-11-16T20:24:26.027 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.027 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.027 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.028 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.726680+0000", 2020-11-16T20:24:26.028 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.2163999999999997e-05 2020-11-16T20:24:26.028 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.028 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.028 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:26.028 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.726732+0000", 2020-11-16T20:24:26.028 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012409737000000001 2020-11-16T20:24:26.029 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.029 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.029 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.029 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.739142+0000", 2020-11-16T20:24:26.029 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000102487 2020-11-16T20:24:26.029 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.029 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.029 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:26.030 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.739245+0000", 2020-11-16T20:24:26.030 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0022202709999999998 2020-11-16T20:24:26.030 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.030 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.030 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.030 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.741465+0000", 2020-11-16T20:24:26.030 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.8485000000000005e-05 2020-11-16T20:24:26.031 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.031 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.031 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:26.031 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.741543+0000", 2020-11-16T20:24:26.031 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.36105948500000001 2020-11-16T20:24:26.031 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.031 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.031 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:26.032 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:16.102603+0000", 2020-11-16T20:24:26.032 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.38311783900000002 2020-11-16T20:24:26.032 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.032 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.032 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.032 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.032 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.032 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:286 3.2s0 3:414e2876:::smithi09216568-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2020-11-16T20:24:26.033 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:25.733778+0000", 2020-11-16T20:24:26.033 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 0.124177545, 2020-11-16T20:24:26.033 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.029899799000000001, 2020-11-16T20:24:26.033 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:26.033 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:26.033 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:26.033 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:26.034 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:26.034 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 286 2020-11-16T20:24:26.034 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.034 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:26.034 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.034 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:26.034 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.733778+0000", 2020-11-16T20:24:26.034 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:26.035 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.035 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.035 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:26.035 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.733778+0000", 2020-11-16T20:24:26.035 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.1762999999999999e-05 2020-11-16T20:24:26.035 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.035 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.035 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:26.036 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.733830+0000", 2020-11-16T20:24:26.036 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.5999999999999996e-06 2020-11-16T20:24:26.036 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.036 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.036 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:26.036 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.733839+0000", 2020-11-16T20:24:26.036 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.8442999999999999e-05 2020-11-16T20:24:26.037 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.037 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.037 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:26.037 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.733858+0000", 2020-11-16T20:24:26.037 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.0169999999999999e-06 2020-11-16T20:24:26.037 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.037 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.037 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:26.038 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.733865+0000", 2020-11-16T20:24:26.038 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00086802400000000001 2020-11-16T20:24:26.038 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.038 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.038 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.038 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.734733+0000", 2020-11-16T20:24:26.038 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00017390799999999999 2020-11-16T20:24:26.038 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.039 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.039 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for blocked object", 2020-11-16T20:24:26.039 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.734907+0000", 2020-11-16T20:24:26.039 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.023992955999999999 2020-11-16T20:24:26.039 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.039 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.039 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.040 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.758900+0000", 2020-11-16T20:24:26.040 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00017244 2020-11-16T20:24:26.040 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.040 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.040 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:26.040 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.759072+0000", 2020-11-16T20:24:26.040 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0040152440000000003 2020-11-16T20:24:26.040 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.041 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.041 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.041 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.763088+0000", 2020-11-16T20:24:26.041 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00025402900000000002 2020-11-16T20:24:26.041 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.041 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.041 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:26.042 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.763342+0000", 2020-11-16T20:24:26.042 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00033637499999999998 2020-11-16T20:24:26.042 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.042 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.042 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:26.042 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.763678+0000", 2020-11-16T20:24:26.042 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.029899799000000001 2020-11-16T20:24:26.042 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.043 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.043 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.043 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.043 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.043 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:292 3.7s0 3:ea3bbe80:::smithi09216568-24:head [append 0~712704] snapc 9=[] ondisk+write+known_if_redirected e41)", 2020-11-16T20:24:26.043 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:25.775070+0000", 2020-11-16T20:24:26.043 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 0.082885249999999994, 2020-11-16T20:24:26.044 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.029551780999999999, 2020-11-16T20:24:26.044 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:26.044 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:26.044 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:26.044 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:26.044 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:26.044 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 292 2020-11-16T20:24:26.044 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.045 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:26.045 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.045 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:26.045 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.775070+0000", 2020-11-16T20:24:26.045 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:26.045 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.045 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.045 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:26.046 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.775070+0000", 2020-11-16T20:24:26.046 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00073570700000000005 2020-11-16T20:24:26.046 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.046 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.046 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:26.046 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.775806+0000", 2020-11-16T20:24:26.046 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.6526999999999999e-05 2020-11-16T20:24:26.047 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.047 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.047 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:26.047 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.775823+0000", 2020-11-16T20:24:26.047 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.7739999999999999e-05 2020-11-16T20:24:26.047 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.047 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.047 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:26.048 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.775850+0000", 2020-11-16T20:24:26.048 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.2614e-05 2020-11-16T20:24:26.048 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.048 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.048 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:26.048 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.775863+0000", 2020-11-16T20:24:26.048 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000102419 2020-11-16T20:24:26.048 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.049 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.049 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.049 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.775965+0000", 2020-11-16T20:24:26.049 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00019994300000000001 2020-11-16T20:24:26.049 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.049 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.049 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:26.050 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.776165+0000", 2020-11-16T20:24:26.050 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0015956849999999999 2020-11-16T20:24:26.050 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.050 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.050 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:26.050 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.777761+0000", 2020-11-16T20:24:26.050 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.016717201000000001 2020-11-16T20:24:26.050 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.051 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.051 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:26.051 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.794478+0000", 2020-11-16T20:24:26.051 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.010026154000000001 2020-11-16T20:24:26.051 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.051 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.051 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:26.052 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.804504+0000", 2020-11-16T20:24:26.052 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000117791 2020-11-16T20:24:26.052 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.052 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.052 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:26.052 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.804622+0000", 2020-11-16T20:24:26.052 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.029551780999999999 2020-11-16T20:24:26.052 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.053 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.053 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.053 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.053 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.053 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:293 3.7s0 3:ea3bbe80:::smithi09216568-24:head [append 0~401408] snapc 9=[] ondisk+write+known_if_redirected e41)", 2020-11-16T20:24:26.053 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:25.777560+0000", 2020-11-16T20:24:26.053 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 0.080395741000000007, 2020-11-16T20:24:26.053 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.031032255000000002, 2020-11-16T20:24:26.054 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:26.054 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:26.054 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:26.054 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:26.054 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:26.054 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 293 2020-11-16T20:24:26.054 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.055 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:26.055 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.055 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:26.055 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.777560+0000", 2020-11-16T20:24:26.055 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:26.055 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.055 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.055 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:26.056 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.777560+0000", 2020-11-16T20:24:26.056 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0010964970000000001 2020-11-16T20:24:26.056 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.056 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.056 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:26.056 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778656+0000", 2020-11-16T20:24:26.056 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4422e-05 2020-11-16T20:24:26.056 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.057 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.057 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:26.057 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778671+0000", 2020-11-16T20:24:26.057 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.5148e-05 2020-11-16T20:24:26.057 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.057 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.057 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:26.058 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778696+0000", 2020-11-16T20:24:26.058 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.1206000000000001e-05 2020-11-16T20:24:26.058 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.058 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.058 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:26.058 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778707+0000", 2020-11-16T20:24:26.058 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0088336160000000007 2020-11-16T20:24:26.058 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.059 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.059 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.059 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.787541+0000", 2020-11-16T20:24:26.059 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00013570699999999999 2020-11-16T20:24:26.059 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.059 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.059 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:26.060 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.787677+0000", 2020-11-16T20:24:26.060 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00081066199999999997 2020-11-16T20:24:26.060 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.060 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.060 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:26.060 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.788487+0000", 2020-11-16T20:24:26.060 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0079055070000000009 2020-11-16T20:24:26.060 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.061 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.061 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:26.061 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.796393+0000", 2020-11-16T20:24:26.061 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012145243 2020-11-16T20:24:26.061 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.061 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.061 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:26.061 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.808538+0000", 2020-11-16T20:24:26.062 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.4246999999999997e-05 2020-11-16T20:24:26.062 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.062 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.062 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:26.062 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.808592+0000", 2020-11-16T20:24:26.062 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.031032255000000002 2020-11-16T20:24:26.062 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.063 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.063 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.063 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.063 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.063 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:294 3.7s0 3:ea3bbe80:::smithi09216568-24:head [setxattr _header (56) in=7b] snapc 9=[] ondisk+write+known_if_redirected e41)", 2020-11-16T20:24:26.063 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:25.778840+0000", 2020-11-16T20:24:26.063 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 0.079115492999999995, 2020-11-16T20:24:26.063 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030227948000000001, 2020-11-16T20:24:26.064 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:26.064 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:26.064 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:26.064 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:26.064 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:26.064 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 294 2020-11-16T20:24:26.064 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.065 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:26.065 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.065 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:26.065 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778840+0000", 2020-11-16T20:24:26.065 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:26.065 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.065 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.065 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:26.066 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778840+0000", 2020-11-16T20:24:26.066 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0083774379999999992 2020-11-16T20:24:26.066 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.066 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.066 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:26.066 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.787218+0000", 2020-11-16T20:24:26.066 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.6339999999999999e-05 2020-11-16T20:24:26.067 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.067 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.067 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:26.067 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.787234+0000", 2020-11-16T20:24:26.067 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.4162e-05 2020-11-16T20:24:26.067 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.067 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.067 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:26.068 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.787258+0000", 2020-11-16T20:24:26.068 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.3655999999999999e-05 2020-11-16T20:24:26.068 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.068 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.068 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:26.068 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.787272+0000", 2020-11-16T20:24:26.068 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0062336809999999996 2020-11-16T20:24:26.068 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.069 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.069 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:26.069 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.793505+0000", 2020-11-16T20:24:26.069 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00011613299999999999 2020-11-16T20:24:26.069 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.069 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.069 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:26.070 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.793622+0000", 2020-11-16T20:24:26.070 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000538361 2020-11-16T20:24:26.070 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.070 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.070 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:26.070 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.794160+0000", 2020-11-16T20:24:26.070 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.002236605 2020-11-16T20:24:26.070 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.071 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.071 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:26.071 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.796397+0000", 2020-11-16T20:24:26.071 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012488011 2020-11-16T20:24:26.071 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.071 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.071 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:26.071 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.808885+0000", 2020-11-16T20:24:26.072 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000183561 2020-11-16T20:24:26.072 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:26.072 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:26.072 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:26.072 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.809068+0000", 2020-11-16T20:24:26.072 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030227948000000001 2020-11-16T20:24:26.072 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.073 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.073 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.073 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:26.073 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:26.073 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:26.074 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-11-16T20:24:26.075 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:26.072+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:26.115 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T20:24:26.175 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:26.173+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:26.204 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:26.204 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:26.204 INFO:teuthology.orchestra.run.smithi013.stdout: "num_ops": 0 2020-11-16T20:24:26.204 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:26.215 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-11-16T20:24:26.276 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:26.272+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:26.288 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:26.317 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:26.335 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:26.335 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:26.335 INFO:teuthology.orchestra.run.smithi013.stdout: "complaint_time": 30, 2020-11-16T20:24:26.335 INFO:teuthology.orchestra.run.smithi013.stdout: "num_blocked_ops": 0 2020-11-16T20:24:26.335 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:26.345 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-11-16T20:24:26.346 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:26.372 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:26.377 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:26.377+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:26.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:26.433 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:26.463 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:26.478 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:26.477+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:26.489 INFO:tasks.rados.rados.0.smithi092.stdout:120: done (5 left) 2020-11-16T20:24:26.490 INFO:tasks.rados.rados.0.smithi092.stdout:121: done (4 left) 2020-11-16T20:24:26.490 INFO:tasks.rados.rados.0.smithi092.stdout:122: done (3 left) 2020-11-16T20:24:26.490 INFO:tasks.rados.rados.0.smithi092.stdout:124: done (2 left) 2020-11-16T20:24:26.490 INFO:tasks.rados.rados.0.smithi092.stdout:128: done (1 left) 2020-11-16T20:24:26.490 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 27 current snap is 6 2020-11-16T20:24:26.490 INFO:tasks.rados.rados.0.smithi092.stdout:117: finishing write tid 3 to smithi09216568-49 2020-11-16T20:24:26.491 INFO:tasks.rados.rados.0.smithi092.stdout:129: seq_num 67 ranges {442368=729088,1171456=401408} 2020-11-16T20:24:26.495 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2020-11-16T20:24:26.497 INFO:tasks.rados.rados.0.smithi092.stdout:129: writing smithi09216568-27 from 442368 to 1171456 tid 1 2020-11-16T20:24:26.497 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:26.499 INFO:tasks.rados.rados.0.smithi092.stdout:129: writing smithi09216568-27 from 1171456 to 1572864 tid 2 2020-11-16T20:24:26.499 INFO:tasks.rados.rados.0.smithi092.stdout:130: delete oid 39 current snap is 6 2020-11-16T20:24:26.501 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T20:24:26.501 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T20:24:26.501 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T20:24:26.501 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-11-16T20:24:26.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-11-16T20:24:26.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-11-16T20:24:26.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-11-16T20:24:26.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-11-16T20:24:26.511 INFO:tasks.rados.rados.0.smithi092.stdout:130: done (2 left) 2020-11-16T20:24:26.511 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 16 current snap is 6 2020-11-16T20:24:26.511 INFO:tasks.rados.rados.0.smithi092.stdout:131: seq_num 69 ranges {253952=761856,1015808=401408} 2020-11-16T20:24:26.514 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2020-11-16T20:24:26.517 INFO:tasks.rados.rados.0.smithi092.stdout:131: writing smithi09216568-16 from 253952 to 1015808 tid 1 2020-11-16T20:24:26.520 INFO:tasks.rados.rados.0.smithi092.stdout:131: writing smithi09216568-16 from 1015808 to 1417216 tid 2 2020-11-16T20:24:26.520 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 33 current snap is 6 2020-11-16T20:24:26.520 INFO:tasks.rados.rados.0.smithi092.stdout:132: seq_num 70 ranges {540672=778240,1318912=401408} 2020-11-16T20:24:26.520 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:26.521 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:26.528 INFO:tasks.rados.rados.0.smithi092.stdout:132: writing smithi09216568-33 from 540672 to 1318912 tid 1 2020-11-16T20:24:26.531 INFO:tasks.rados.rados.0.smithi092.stdout:132: writing smithi09216568-33 from 1318912 to 1720320 tid 2 2020-11-16T20:24:26.532 INFO:tasks.rados.rados.0.smithi092.stdout:133: delete oid 30 current snap is 6 2020-11-16T20:24:26.532 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-11-16T20:24:26.533 INFO:tasks.rados.rados.0.smithi092.stdout:129: finishing write tid 1 to smithi09216568-27 2020-11-16T20:24:26.533 INFO:tasks.rados.rados.0.smithi092.stdout:129: finishing write tid 2 to smithi09216568-27 2020-11-16T20:24:26.533 INFO:tasks.rados.rados.0.smithi092.stdout:129: finishing write tid 3 to smithi09216568-27 2020-11-16T20:24:26.534 INFO:tasks.rados.rados.0.smithi092.stdout:129: finishing write tid 4 to smithi09216568-27 2020-11-16T20:24:26.534 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 27 v 16 (ObjNum 67 snap 6 seq_num 67) dirty exists 2020-11-16T20:24:26.534 INFO:tasks.rados.rados.0.smithi092.stdout:129: left oid 27 (ObjNum 67 snap 6 seq_num 67) 2020-11-16T20:24:26.537 INFO:tasks.rados.rados.0.smithi092.stdout:131: finishing write tid 1 to smithi09216568-16 2020-11-16T20:24:26.538 INFO:tasks.rados.rados.0.smithi092.stdout:131: finishing write tid 2 to smithi09216568-16 2020-11-16T20:24:26.539 INFO:tasks.rados.rados.0.smithi092.stdout:131: finishing write tid 3 to smithi09216568-16 2020-11-16T20:24:26.542 INFO:tasks.rados.rados.0.smithi092.stdout:131: finishing write tid 4 to smithi09216568-16 2020-11-16T20:24:26.542 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 16 v 25 (ObjNum 69 snap 6 seq_num 69) dirty exists 2020-11-16T20:24:26.542 INFO:tasks.rados.rados.0.smithi092.stdout:131: left oid 16 (ObjNum 69 snap 6 seq_num 69) 2020-11-16T20:24:26.553 INFO:tasks.rados.rados.0.smithi092.stdout:132: finishing write tid 1 to smithi09216568-33 2020-11-16T20:24:26.556 INFO:tasks.rados.rados.0.smithi092.stdout:132: finishing write tid 2 to smithi09216568-33 2020-11-16T20:24:26.557 INFO:tasks.rados.rados.0.smithi092.stdout:132: finishing write tid 3 to smithi09216568-33 2020-11-16T20:24:26.557 INFO:tasks.rados.rados.0.smithi092.stdout:129: done (4 left) 2020-11-16T20:24:26.558 INFO:tasks.rados.rados.0.smithi092.stdout:131: done (3 left) 2020-11-16T20:24:26.558 INFO:tasks.rados.rados.0.smithi092.stdout:133: done (2 left) 2020-11-16T20:24:26.558 INFO:tasks.rados.rados.0.smithi092.stdout:134: delete oid 17 current snap is 6 2020-11-16T20:24:26.558 INFO:tasks.rados.rados.0.smithi092.stdout:132: finishing write tid 4 to smithi09216568-33 2020-11-16T20:24:26.558 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 33 v 9 (ObjNum 70 snap 6 seq_num 70) dirty exists 2020-11-16T20:24:26.559 INFO:tasks.rados.rados.0.smithi092.stdout:132: left oid 33 (ObjNum 70 snap 6 seq_num 70) 2020-11-16T20:24:26.562 INFO:tasks.rados.rados.0.smithi092.stdout:132: done (2 left) 2020-11-16T20:24:26.562 INFO:tasks.rados.rados.0.smithi092.stdout:134: done (1 left) 2020-11-16T20:24:26.563 INFO:tasks.rados.rados.0.smithi092.stdout:135: rmattr oid 8 current snap is 6 2020-11-16T20:24:26.563 INFO:tasks.rados.rados.0.smithi092.stdout:135: done (1 left) 2020-11-16T20:24:26.563 INFO:tasks.rados.rados.0.smithi092.stdout:136: rollback oid 11 current snap is 6 2020-11-16T20:24:26.563 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 11 to 5 2020-11-16T20:24:26.563 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 42 current snap is 6 2020-11-16T20:24:26.563 INFO:tasks.rados.rados.0.smithi092.stdout:137: seq_num 73 ranges {696320=425984,1122304=409600,1531904=393216} 2020-11-16T20:24:26.570 INFO:tasks.rados.rados.0.smithi092.stdout:137: writing smithi09216568-42 from 696320 to 1122304 tid 1 2020-11-16T20:24:26.578 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:26.577+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:26.579 INFO:tasks.rados.rados.0.smithi092.stdout:137: writing smithi09216568-42 from 1122304 to 1531904 tid 2 2020-11-16T20:24:26.582 INFO:tasks.rados.rados.0.smithi092.stdout:137: writing smithi09216568-42 from 1531904 to 1925120 tid 3 2020-11-16T20:24:26.582 INFO:tasks.rados.rados.0.smithi092.stdout:138: delete oid 27 current snap is 6 2020-11-16T20:24:26.582 INFO:tasks.rados.rados.0.smithi092.stdout:136: finishing rollback tid 0 to smithi09216568-11 2020-11-16T20:24:26.582 INFO:tasks.rados.rados.0.smithi092.stdout:136: finishing rollback tid 1 to smithi09216568-11 2020-11-16T20:24:26.582 INFO:tasks.rados.rados.0.smithi092.stdout:136: finishing rollback tid 2 to smithi09216568-11 2020-11-16T20:24:26.583 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 39 (ObjNum 59 snap 5 seq_num 59) dirty exists 2020-11-16T20:24:26.586 INFO:tasks.rados.rados.0.smithi092.stdout:136: done (3 left) 2020-11-16T20:24:26.586 INFO:tasks.rados.rados.0.smithi092.stdout:138: done (2 left) 2020-11-16T20:24:26.586 INFO:tasks.rados.rados.0.smithi092.stdout:139: setattr oid 5 current snap is 6 2020-11-16T20:24:26.589 INFO:tasks.rados.rados.0.smithi092.stdout:140: rollback oid 41 current snap is 6 2020-11-16T20:24:26.590 INFO:tasks.rados.rados.0.smithi092.stdout:rollback oid 41 to 5 2020-11-16T20:24:26.590 INFO:tasks.rados.rados.0.smithi092.stdout:141: copy_from oid 19 from oid 24 current snap is 6 2020-11-16T20:24:26.590 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 16 current snap is 6 2020-11-16T20:24:26.590 INFO:tasks.rados.rados.0.smithi092.stdout:142: seq_num 75 ranges {1417216=458752,1875968=581632,2457600=221184} 2020-11-16T20:24:26.601 INFO:tasks.rados.rados.0.smithi092.stdout:142: writing smithi09216568-16 from 1417216 to 1875968 tid 1 2020-11-16T20:24:26.605 INFO:tasks.rados.rados.0.smithi092.stdout:142: writing smithi09216568-16 from 1875968 to 2457600 tid 2 2020-11-16T20:24:26.606 INFO:tasks.rados.rados.0.smithi092.stdout:142: writing smithi09216568-16 from 2457600 to 2678784 tid 3 2020-11-16T20:24:26.606 INFO:tasks.rados.rados.0.smithi092.stdout:143: delete oid 10 current snap is 6 2020-11-16T20:24:26.607 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 1 to smithi09216568-42 2020-11-16T20:24:26.607 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 2 to smithi09216568-42 2020-11-16T20:24:26.607 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 3 to smithi09216568-42 2020-11-16T20:24:26.607 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 4 to smithi09216568-42 2020-11-16T20:24:26.607 INFO:tasks.rados.rados.0.smithi092.stdout:137: finishing write tid 5 to smithi09216568-42 2020-11-16T20:24:26.607 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 42 v 15 (ObjNum 73 snap 6 seq_num 73) dirty exists 2020-11-16T20:24:26.608 INFO:tasks.rados.rados.0.smithi092.stdout:137: left oid 42 (ObjNum 73 snap 6 seq_num 73) 2020-11-16T20:24:26.608 INFO:tasks.rados.rados.0.smithi092.stdout:140: finishing rollback tid 0 to smithi09216568-41 2020-11-16T20:24:26.608 INFO:tasks.rados.rados.0.smithi092.stdout:140: finishing rollback tid 1 to smithi09216568-41 2020-11-16T20:24:26.608 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 5 v 41 (ObjNum 4 snap 0 seq_num 4) dirty exists 2020-11-16T20:24:26.608 INFO:tasks.rados.rados.0.smithi092.stdout:140: finishing rollback tid 2 to smithi09216568-41 2020-11-16T20:24:26.608 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 41 v 36 (ObjNum 40 snap 0 seq_num 40) dirty exists 2020-11-16T20:24:26.618 INFO:tasks.rados.rados.0.smithi092.stdout:137: done (6 left) 2020-11-16T20:24:26.618 INFO:tasks.rados.rados.0.smithi092.stdout:139: done (5 left) 2020-11-16T20:24:26.619 INFO:tasks.rados.rados.0.smithi092.stdout:140: done (4 left) 2020-11-16T20:24:26.619 INFO:tasks.rados.rados.0.smithi092.stdout:143: done (3 left) 2020-11-16T20:24:26.619 INFO:tasks.rados.rados.0.smithi092.stdout:144: delete oid 41 current snap is 6 2020-11-16T20:24:26.625 INFO:tasks.rados.rados.0.smithi092.stdout:144: done (3 left) 2020-11-16T20:24:26.625 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 33 current snap is 6 2020-11-16T20:24:26.625 INFO:tasks.rados.rados.0.smithi092.stdout:145: seq_num 78 ranges {1720320=516096,2236416=679936,2916352=122880} 2020-11-16T20:24:26.642 INFO:tasks.rados.rados.0.smithi092.stdout:145: writing smithi09216568-33 from 1720320 to 2236416 tid 1 2020-11-16T20:24:26.643 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:26.640+0000 7f7371c75700 -1 osd.6 pg_epoch: 42 pg[3.4s0( v 41'22 (0'0,41'22] local-lis/les=23/24 n=6 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) [6,5,4]p6(0) r=0 lpr=23 crt=41'22 lcod 41'21 mlcod 41'21 active+clean+snaptrim trimq=[9~1] ps=[3~2,6~1]] removing snap head 2020-11-16T20:24:26.646 INFO:tasks.rados.rados.0.smithi092.stdout:145: writing smithi09216568-33 from 2236416 to 2916352 tid 2 2020-11-16T20:24:26.647 INFO:tasks.rados.rados.0.smithi092.stdout:145: writing smithi09216568-33 from 2916352 to 3039232 tid 3 2020-11-16T20:24:26.648 INFO:tasks.rados.rados.0.smithi092.stdout:141: finishing copy_from to smithi09216568-19 2020-11-16T20:24:26.648 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 19 v 31 (ObjNum 66 snap 5 seq_num 66) dirty exists 2020-11-16T20:24:26.648 INFO:tasks.rados.rados.0.smithi092.stdout:141: finishing copy_from racing read to smithi09216568-19 2020-11-16T20:24:26.648 INFO:tasks.rados.rados.0.smithi092.stdout:142: finishing write tid 1 to smithi09216568-16 2020-11-16T20:24:26.648 INFO:tasks.rados.rados.0.smithi092.stdout:142: finishing write tid 2 to smithi09216568-16 2020-11-16T20:24:26.648 INFO:tasks.rados.rados.0.smithi092.stdout:142: finishing write tid 3 to smithi09216568-16 2020-11-16T20:24:26.649 INFO:tasks.rados.rados.0.smithi092.stdout:117: finishing write tid 4 to smithi09216568-49 2020-11-16T20:24:26.649 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 49 v 22 (ObjNum 65 snap 5 seq_num 65) dirty exists 2020-11-16T20:24:26.649 INFO:tasks.rados.rados.0.smithi092.stdout:117: left oid 49 (ObjNum 65 snap 5 seq_num 65) 2020-11-16T20:24:26.649 INFO:tasks.rados.rados.0.smithi092.stdout:117: done (3 left) 2020-11-16T20:24:26.649 INFO:tasks.rados.rados.0.smithi092.stdout:141: done (2 left) 2020-11-16T20:24:26.649 INFO:tasks.rados.rados.0.smithi092.stdout:146: snap_create 2020-11-16T20:24:26.665 INFO:tasks.rados.rados.0.smithi092.stdout:145: finishing write tid 1 to smithi09216568-33 2020-11-16T20:24:26.665 INFO:tasks.rados.rados.0.smithi092.stdout:145: finishing write tid 2 to smithi09216568-33 2020-11-16T20:24:26.666 INFO:tasks.rados.rados.0.smithi092.stdout:145: finishing write tid 3 to smithi09216568-33 2020-11-16T20:24:26.668 INFO:tasks.rados.rados.0.smithi092.stdout:145: finishing write tid 4 to smithi09216568-33 2020-11-16T20:24:26.670 INFO:tasks.rados.rados.0.smithi092.stdout:145: finishing write tid 5 to smithi09216568-33 2020-11-16T20:24:26.670 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 33 v 15 (ObjNum 78 snap 6 seq_num 78) dirty exists 2020-11-16T20:24:26.670 INFO:tasks.rados.rados.0.smithi092.stdout:145: left oid 33 (ObjNum 78 snap 6 seq_num 78) 2020-11-16T20:24:26.672 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:26.673 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:26.679 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:26.676+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:26.683 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-11-16T20:24:26.727 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2020-11-16T20:24:26.779 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:26.776+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:26.854 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:26.854 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:26.865 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-11-16T20:24:26.881 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:26.881+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:26.981 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:26.980+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:27.029 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:27.030 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:27.040 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-11-16T20:24:27.082 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:27.081+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:27.183 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:27.180+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:27.201 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:27.202 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:27.212 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-11-16T20:24:27.284 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:27.285+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:27.376 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:27.376 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:27.385 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:27.385+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:27.386 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-11-16T20:24:27.487 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:27.484+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:27.504 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:27.505 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:27.515 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T20:24:27.560 INFO:tasks.rados.rados.0.smithi092.stdout:142: finishing write tid 4 to smithi09216568-16 2020-11-16T20:24:27.564 INFO:tasks.rados.rados.0.smithi092.stdout:142: finishing write tid 5 to smithi09216568-16 2020-11-16T20:24:27.564 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 16 v 35 (ObjNum 75 snap 6 seq_num 75) dirty exists 2020-11-16T20:24:27.564 INFO:tasks.rados.rados.0.smithi092.stdout:142: left oid 16 (ObjNum 75 snap 6 seq_num 75) 2020-11-16T20:24:27.588 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:27.584+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:27.642 INFO:tasks.rados.rados.0.smithi092.stdout:142: done (2 left) 2020-11-16T20:24:27.642 INFO:tasks.rados.rados.0.smithi092.stdout:145: done (1 left) 2020-11-16T20:24:27.642 INFO:tasks.rados.rados.0.smithi092.stdout:146: done (0 left) 2020-11-16T20:24:27.642 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 3 current snap is 7 2020-11-16T20:24:27.642 INFO:tasks.rados.rados.0.smithi092.stdout:147: seq_num 79 ranges {40960=532480,573440=565248,1138688=237568} 2020-11-16T20:24:27.646 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2020-11-16T20:24:27.647 INFO:tasks.rados.rados.0.smithi092.stdout:147: writing smithi09216568-3 from 40960 to 573440 tid 1 2020-11-16T20:24:27.649 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2020-11-16T20:24:27.650 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:27.650 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:27.651 INFO:tasks.rados.rados.0.smithi092.stdout:147: writing smithi09216568-3 from 573440 to 1138688 tid 2 2020-11-16T20:24:27.659 INFO:tasks.rados.rados.0.smithi092.stdout:147: writing smithi09216568-3 from 1138688 to 1376256 tid 3 2020-11-16T20:24:27.659 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 39 current snap is 7 2020-11-16T20:24:27.660 INFO:tasks.rados.rados.0.smithi092.stdout:148: seq_num 80 ranges {0=548864,548864=450560,999424=352256} 2020-11-16T20:24:27.663 INFO:tasks.rados.rados.0.smithi092.stdout:148: writing smithi09216568-39 from 0 to 548864 tid 1 2020-11-16T20:24:27.666 INFO:tasks.rados.rados.0.smithi092.stdout:148: writing smithi09216568-39 from 548864 to 999424 tid 2 2020-11-16T20:24:27.667 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T20:24:27.668 INFO:tasks.rados.rados.0.smithi092.stdout:148: writing smithi09216568-39 from 999424 to 1351680 tid 3 2020-11-16T20:24:27.669 INFO:tasks.rados.rados.0.smithi092.stdout:149: copy_from oid 32 from oid 42 current snap is 7 2020-11-16T20:24:27.669 INFO:tasks.rados.rados.0.smithi092.stdout:150: snap_create 2020-11-16T20:24:27.672 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2020-11-16T20:24:27.674 INFO:tasks.rados.rados.0.smithi092.stdout:147: finishing write tid 1 to smithi09216568-3 2020-11-16T20:24:27.677 INFO:tasks.rados.rados.0.smithi092.stdout:147: finishing write tid 2 to smithi09216568-3 2020-11-16T20:24:27.678 INFO:tasks.rados.rados.0.smithi092.stdout:147: finishing write tid 3 to smithi09216568-3 2020-11-16T20:24:27.678 INFO:tasks.rados.rados.0.smithi092.stdout:147: finishing write tid 4 to smithi09216568-3 2020-11-16T20:24:27.683 INFO:tasks.rados.rados.0.smithi092.stdout:147: finishing write tid 5 to smithi09216568-3 2020-11-16T20:24:27.683 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 3 v 26 (ObjNum 79 snap 7 seq_num 79) dirty exists 2020-11-16T20:24:27.683 INFO:tasks.rados.rados.0.smithi092.stdout:147: left oid 3 (ObjNum 79 snap 7 seq_num 79) 2020-11-16T20:24:27.688 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:27.689+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:27.699 INFO:tasks.rados.rados.0.smithi092.stdout:148: finishing write tid 1 to smithi09216568-39 2020-11-16T20:24:27.700 INFO:tasks.rados.rados.0.smithi092.stdout:148: finishing write tid 2 to smithi09216568-39 2020-11-16T20:24:27.702 INFO:tasks.rados.rados.0.smithi092.stdout:148: finishing write tid 3 to smithi09216568-39 2020-11-16T20:24:27.702 INFO:tasks.rados.rados.0.smithi092.stdout:148: finishing write tid 4 to smithi09216568-39 2020-11-16T20:24:27.704 INFO:tasks.rados.rados.0.smithi092.stdout:148: finishing write tid 5 to smithi09216568-39 2020-11-16T20:24:27.704 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 39 v 29 (ObjNum 80 snap 7 seq_num 80) dirty exists 2020-11-16T20:24:27.704 INFO:tasks.rados.rados.0.smithi092.stdout:148: left oid 39 (ObjNum 80 snap 7 seq_num 80) 2020-11-16T20:24:27.708 INFO:tasks.rados.rados.0.smithi092.stdout:149: finishing copy_from to smithi09216568-32 2020-11-16T20:24:27.708 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 32 v 15 (ObjNum 73 snap 6 seq_num 73) dirty exists 2020-11-16T20:24:27.709 INFO:tasks.rados.rados.0.smithi092.stdout:149: finishing copy_from racing read to smithi09216568-32 2020-11-16T20:24:27.780 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:27.781 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:27.789 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:27.788+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:27.791 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T20:24:27.890 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:27.888+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:27.944 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:27.945 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:27.955 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T20:24:27.990 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:27.988+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:28.069 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:28.069 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:28.079 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T20:24:28.124 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:28.125+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:28.192 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:28.188+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:28.197 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:28.198 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:28.208 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T20:24:28.293 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:28.292+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:28.374 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:28.375 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:28.385 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T20:24:28.393 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:28.392+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:28.494 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:28.493+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:28.504 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:28.504 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:28.514 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T20:24:28.518 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T20:24:28.595 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:28.592+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:28.655 INFO:tasks.rados.rados.0.smithi092.stdout:147: done (3 left) 2020-11-16T20:24:28.656 INFO:tasks.rados.rados.0.smithi092.stdout:148: done (2 left) 2020-11-16T20:24:28.656 INFO:tasks.rados.rados.0.smithi092.stdout:149: done (1 left) 2020-11-16T20:24:28.656 INFO:tasks.rados.rados.0.smithi092.stdout:150: done (0 left) 2020-11-16T20:24:28.656 INFO:tasks.rados.rados.0.smithi092.stdout:151: setattr oid 42 current snap is 8 2020-11-16T20:24:28.657 INFO:tasks.rados.rados.0.smithi092.stdout:152: delete oid 12 current snap is 8 2020-11-16T20:24:28.664 INFO:tasks.rados.rados.0.smithi092.stdout:152: done (1 left) 2020-11-16T20:24:28.665 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 46 current snap is 8 2020-11-16T20:24:28.670 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2020-11-16T20:24:28.670 INFO:tasks.rados.rados.0.smithi092.stdout:153: seq_num 82 ranges {761856=581632,1343488=622592,1966080=180224} 2020-11-16T20:24:28.674 INFO:tasks.rados.rados.0.smithi092.stdout:153: writing smithi09216568-46 from 761856 to 1343488 tid 1 2020-11-16T20:24:28.677 INFO:tasks.rados.rados.0.smithi092.stdout:153: writing smithi09216568-46 from 1343488 to 1966080 tid 2 2020-11-16T20:24:28.678 INFO:tasks.rados.rados.0.smithi092.stdout:153: writing smithi09216568-46 from 1966080 to 2146304 tid 3 2020-11-16T20:24:28.679 INFO:tasks.rados.rados.0.smithi092.stdout:154: snap_create 2020-11-16T20:24:28.679 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 42 v 17 (ObjNum 73 snap 6 seq_num 73) dirty exists 2020-11-16T20:24:28.681 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-11-16T20:24:28.681 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T20:24:28.696 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:28.696 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:28.696 INFO:tasks.rados.rados.0.smithi092.stdout:153: finishing write tid 1 to smithi09216568-46 2020-11-16T20:24:28.697 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:28.697+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:28.697 INFO:tasks.rados.rados.0.smithi092.stdout:153: finishing write tid 2 to smithi09216568-46 2020-11-16T20:24:28.697 INFO:tasks.rados.rados.0.smithi092.stdout:153: finishing write tid 3 to smithi09216568-46 2020-11-16T20:24:28.699 INFO:tasks.rados.rados.0.smithi092.stdout:153: finishing write tid 4 to smithi09216568-46 2020-11-16T20:24:28.707 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T20:24:28.717 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:28.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:28.768 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:28.795 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:28.797 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:28.797+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:28.825 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:28.837 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:28.837 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:28.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:28.838 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:28.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.838 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 30/26 ECSubWrite(tid=0, reqid=client.4444.0:1, at_version=30'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:28.838 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:28.838 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.304915465000001, 2020-11-16T20:24:28.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110371539, 2020-11-16T20:24:28.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:28.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:28.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532879+0000", 2020-11-16T20:24:28.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1965999999999994e-05 2020-11-16T20:24:28.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532961+0000", 2020-11-16T20:24:28.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4882000000000002e-05 2020-11-16T20:24:28.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532996+0000", 2020-11-16T20:24:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.107939761 2020-11-16T20:24:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640936+0000", 2020-11-16T20:24:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3749999999999997e-06 2020-11-16T20:24:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640943+0000", 2020-11-16T20:24:28.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000229192 2020-11-16T20:24:28.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641172+0000", 2020-11-16T20:24:28.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3837e-05 2020-11-16T20:24:28.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641186+0000", 2020-11-16T20:24:28.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002004053 2020-11-16T20:24:28.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643190+0000", 2020-11-16T20:24:28.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0473000000000002e-05 2020-11-16T20:24:28.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643251+0000", 2020-11-16T20:24:28.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110371539 2020-11-16T20:24:28.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.846 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.847 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:54 3.5s0 3:aad7141b:::smithi09216568-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:28.847 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:28.847 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.263720751999999, 2020-11-16T20:24:28.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027636214999999999, 2020-11-16T20:24:28.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:28.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 54 2020-11-16T20:24:28.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.848 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:28.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574074+0000", 2020-11-16T20:24:28.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7353e-05 2020-11-16T20:24:28.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574111+0000", 2020-11-16T20:24:28.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1509999999999998e-06 2020-11-16T20:24:28.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574118+0000", 2020-11-16T20:24:28.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2278e-05 2020-11-16T20:24:28.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574131+0000", 2020-11-16T20:24:28.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3789999999999997e-06 2020-11-16T20:24:28.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574136+0000", 2020-11-16T20:24:28.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054445750000000001 2020-11-16T20:24:28.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.579581+0000", 2020-11-16T20:24:28.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117794 2020-11-16T20:24:28.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.579698+0000", 2020-11-16T20:24:28.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056929220000000004 2020-11-16T20:24:28.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.585391+0000", 2020-11-16T20:24:28.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106148 2020-11-16T20:24:28.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.585498+0000", 2020-11-16T20:24:28.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097071099999999997 2020-11-16T20:24:28.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.595205+0000", 2020-11-16T20:24:28.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013155099999999999 2020-11-16T20:24:28.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.595336+0000", 2020-11-16T20:24:28.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063739540000000003 2020-11-16T20:24:28.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601710+0000", 2020-11-16T20:24:28.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027636214999999999 2020-11-16T20:24:28.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.857 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.858 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:59 3.es0 3:746783df:::smithi09216568-20:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:28.858 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:28.858 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.263253621, 2020-11-16T20:24:28.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026665233, 2020-11-16T20:24:28.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:28.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59 2020-11-16T20:24:28.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.860 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:28.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574541+0000", 2020-11-16T20:24:28.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5383999999999997e-05 2020-11-16T20:24:28.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574596+0000", 2020-11-16T20:24:28.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999998e-06 2020-11-16T20:24:28.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574604+0000", 2020-11-16T20:24:28.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3088000000000001e-05 2020-11-16T20:24:28.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574617+0000", 2020-11-16T20:24:28.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1129999999999997e-06 2020-11-16T20:24:28.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574623+0000", 2020-11-16T20:24:28.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007628709 2020-11-16T20:24:28.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582252+0000", 2020-11-16T20:24:28.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011149099999999999 2020-11-16T20:24:28.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582363+0000", 2020-11-16T20:24:28.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069613909999999999 2020-11-16T20:24:28.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589325+0000", 2020-11-16T20:24:28.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0105000000000001e-05 2020-11-16T20:24:28.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589385+0000", 2020-11-16T20:24:28.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052031099999999997 2020-11-16T20:24:28.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.589905+0000", 2020-11-16T20:24:28.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072848720000000004 2020-11-16T20:24:28.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.597190+0000", 2020-11-16T20:24:28.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00390829 2020-11-16T20:24:28.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:28.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601098+0000", 2020-11-16T20:24:28.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108199 2020-11-16T20:24:28.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601206+0000", 2020-11-16T20:24:28.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026665233 2020-11-16T20:24:28.869 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.869 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.869 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.870 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:60 3.es0 3:746783df:::smithi09216568-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:28.870 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:28.870 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.263121679999999, 2020-11-16T20:24:28.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029540154999999998, 2020-11-16T20:24:28.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:28.871 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.871 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.871 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.871 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 60 2020-11-16T20:24:28.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.871 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:28.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574673+0000", 2020-11-16T20:24:28.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9110000000000003e-05 2020-11-16T20:24:28.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574712+0000", 2020-11-16T20:24:28.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8739999999999997e-06 2020-11-16T20:24:28.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574719+0000", 2020-11-16T20:24:28.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2554e-05 2020-11-16T20:24:28.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574732+0000", 2020-11-16T20:24:28.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-06 2020-11-16T20:24:28.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.574736+0000", 2020-11-16T20:24:28.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077321949999999999 2020-11-16T20:24:28.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582469+0000", 2020-11-16T20:24:28.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010602 2020-11-16T20:24:28.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.582575+0000", 2020-11-16T20:24:28.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0078916549999999992 2020-11-16T20:24:28.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.590466+0000", 2020-11-16T20:24:28.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107569 2020-11-16T20:24:28.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.590574+0000", 2020-11-16T20:24:28.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010689094999999999 2020-11-16T20:24:28.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601263+0000", 2020-11-16T20:24:28.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7796999999999999e-05 2020-11-16T20:24:28.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.601361+0000", 2020-11-16T20:24:28.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028524959999999999 2020-11-16T20:24:28.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604213+0000", 2020-11-16T20:24:28.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029540154999999998 2020-11-16T20:24:28.880 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.880 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.881 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s2 31/23 ECSubWrite(tid=15, reqid=client.4444.0:85, at_version=31'7, trim_to=0'0, roll_forward_to=31'5))", 2020-11-16T20:24:28.881 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:28.881 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.232018386, 2020-11-16T20:24:28.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036849253999999998, 2020-11-16T20:24:28.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:28.882 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:28.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605776+0000", 2020-11-16T20:24:28.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014849700000000001 2020-11-16T20:24:28.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605925+0000", 2020-11-16T20:24:28.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2458000000000001e-05 2020-11-16T20:24:28.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605947+0000", 2020-11-16T20:24:28.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5471999999999999e-05 2020-11-16T20:24:28.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606013+0000", 2020-11-16T20:24:28.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8510000000000001e-06 2020-11-16T20:24:28.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606020+0000", 2020-11-16T20:24:28.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000222365 2020-11-16T20:24:28.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606242+0000", 2020-11-16T20:24:28.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8460000000000003e-06 2020-11-16T20:24:28.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.606252+0000", 2020-11-16T20:24:28.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036144568000000002 2020-11-16T20:24:28.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642396+0000", 2020-11-16T20:24:28.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022919699999999999 2020-11-16T20:24:28.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642626+0000", 2020-11-16T20:24:28.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036849253999999998 2020-11-16T20:24:28.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.890 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:112 3.5s0 3:aa220dd4:::smithi09216568-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:28.890 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:28.890 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.197774667999999, 2020-11-16T20:24:28.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033427307000000003, 2020-11-16T20:24:28.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:28.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.891 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 112 2020-11-16T20:24:28.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.891 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:28.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.640020+0000", 2020-11-16T20:24:28.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002370661 2020-11-16T20:24:28.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642391+0000", 2020-11-16T20:24:28.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3389999999999993e-06 2020-11-16T20:24:28.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642399+0000", 2020-11-16T20:24:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.401e-05 2020-11-16T20:24:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642413+0000", 2020-11-16T20:24:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7309999999999996e-06 2020-11-16T20:24:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642420+0000", 2020-11-16T20:24:28.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0634000000000003e-05 2020-11-16T20:24:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642480+0000", 2020-11-16T20:24:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021235000000000001 2020-11-16T20:24:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.642693+0000", 2020-11-16T20:24:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010232970000000001 2020-11-16T20:24:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.643716+0000", 2020-11-16T20:24:28.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018482157999999999 2020-11-16T20:24:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662198+0000", 2020-11-16T20:24:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011105830000000001 2020-11-16T20:24:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673304+0000", 2020-11-16T20:24:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014329699999999999 2020-11-16T20:24:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673447+0000", 2020-11-16T20:24:28.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033427307000000003 2020-11-16T20:24:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:113 3.5s0 3:aa220dd4:::smithi09216568-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:28.900 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.185503502, 2020-11-16T20:24:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035489453999999997, 2020-11-16T20:24:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 113 2020-11-16T20:24:28.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652291+0000", 2020-11-16T20:24:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6527000000000003e-05 2020-11-16T20:24:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652348+0000", 2020-11-16T20:24:28.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7620000000000001e-06 2020-11-16T20:24:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652356+0000", 2020-11-16T20:24:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4928e-05 2020-11-16T20:24:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652371+0000", 2020-11-16T20:24:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0709999999999997e-06 2020-11-16T20:24:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652378+0000", 2020-11-16T20:24:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2488000000000003e-05 2020-11-16T20:24:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652431+0000", 2020-11-16T20:24:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107941 2020-11-16T20:24:28.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652539+0000", 2020-11-16T20:24:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021009645 2020-11-16T20:24:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673549+0000", 2020-11-16T20:24:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010424 2020-11-16T20:24:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.673653+0000", 2020-11-16T20:24:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000577025 2020-11-16T20:24:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674230+0000", 2020-11-16T20:24:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061432429999999996 2020-11-16T20:24:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.680373+0000", 2020-11-16T20:24:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002934313 2020-11-16T20:24:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.683307+0000", 2020-11-16T20:24:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004473271 2020-11-16T20:24:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.687781+0000", 2020-11-16T20:24:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035489453999999997 2020-11-16T20:24:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:114 3.5s0 3:aa220dd4:::smithi09216568-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.185197665, 2020-11-16T20:24:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037793588000000003, 2020-11-16T20:24:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:28.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 114 2020-11-16T20:24:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652597+0000", 2020-11-16T20:24:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5556000000000003e-05 2020-11-16T20:24:28.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652633+0000", 2020-11-16T20:24:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2020-11-16T20:24:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652640+0000", 2020-11-16T20:24:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3174e-05 2020-11-16T20:24:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652653+0000", 2020-11-16T20:24:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1580000000000002e-06 2020-11-16T20:24:28.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652659+0000", 2020-11-16T20:24:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.651e-05 2020-11-16T20:24:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652706+0000", 2020-11-16T20:24:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6873000000000006e-05 2020-11-16T20:24:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652802+0000", 2020-11-16T20:24:28.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021707194999999999 2020-11-16T20:24:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674510+0000", 2020-11-16T20:24:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114652 2020-11-16T20:24:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674624+0000", 2020-11-16T20:24:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013244752 2020-11-16T20:24:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.687869+0000", 2020-11-16T20:24:28.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000157813 2020-11-16T20:24:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688027+0000", 2020-11-16T20:24:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023638050000000001 2020-11-16T20:24:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.690391+0000", 2020-11-16T20:24:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037793588000000003 2020-11-16T20:24:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:115 3.5s0 3:ade97634:::smithi09216568-39:head [append 0~638976,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.184915259, 2020-11-16T20:24:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028710981, 2020-11-16T20:24:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 115 2020-11-16T20:24:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652879+0000", 2020-11-16T20:24:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046393399999999999 2020-11-16T20:24:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653343+0000", 2020-11-16T20:24:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2337000000000001e-05 2020-11-16T20:24:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653356+0000", 2020-11-16T20:24:28.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7708000000000001e-05 2020-11-16T20:24:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653373+0000", 2020-11-16T20:24:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9990000000000008e-06 2020-11-16T20:24:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653381+0000", 2020-11-16T20:24:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2762000000000001e-05 2020-11-16T20:24:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653434+0000", 2020-11-16T20:24:28.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000220905 2020-11-16T20:24:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653655+0000", 2020-11-16T20:24:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001846164 2020-11-16T20:24:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.655501+0000", 2020-11-16T20:24:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007675594 2020-11-16T20:24:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663177+0000", 2020-11-16T20:24:28.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018319185000000002 2020-11-16T20:24:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681496+0000", 2020-11-16T20:24:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4393000000000002e-05 2020-11-16T20:24:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681590+0000", 2020-11-16T20:24:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028710981 2020-11-16T20:24:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:116 3.5s0 3:ade97634:::smithi09216568-39:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.183895637999999, 2020-11-16T20:24:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035068554000000002, 2020-11-16T20:24:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:28.933 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 116 2020-11-16T20:24:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653899+0000", 2020-11-16T20:24:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6311999999999999e-05 2020-11-16T20:24:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653955+0000", 2020-11-16T20:24:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9060000000000008e-06 2020-11-16T20:24:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653964+0000", 2020-11-16T20:24:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3818000000000001e-05 2020-11-16T20:24:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653978+0000", 2020-11-16T20:24:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.755e-06 2020-11-16T20:24:28.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653984+0000", 2020-11-16T20:24:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079596330000000007 2020-11-16T20:24:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.661943+0000", 2020-11-16T20:24:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012344799999999999 2020-11-16T20:24:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662067+0000", 2020-11-16T20:24:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019626787999999999 2020-11-16T20:24:28.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681694+0000", 2020-11-16T20:24:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107509 2020-11-16T20:24:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.681801+0000", 2020-11-16T20:24:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000509633 2020-11-16T20:24:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682311+0000", 2020-11-16T20:24:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061688259999999996 2020-11-16T20:24:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688480+0000", 2020-11-16T20:24:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4555999999999995e-05 2020-11-16T20:24:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688564+0000", 2020-11-16T20:24:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040337000000000001 2020-11-16T20:24:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.688968+0000", 2020-11-16T20:24:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.035068554000000002 2020-11-16T20:24:28.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:117 3.5s0 3:ade97634:::smithi09216568-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:28.944 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.183764952000001, 2020-11-16T20:24:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037529909, 2020-11-16T20:24:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 117 2020-11-16T20:24:28.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654030+0000", 2020-11-16T20:24:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7752e-05 2020-11-16T20:24:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654067+0000", 2020-11-16T20:24:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8970000000000002e-06 2020-11-16T20:24:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654074+0000", 2020-11-16T20:24:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2906e-05 2020-11-16T20:24:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654087+0000", 2020-11-16T20:24:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.181e-06 2020-11-16T20:24:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654092+0000", 2020-11-16T20:24:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080038750000000006 2020-11-16T20:24:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662096+0000", 2020-11-16T20:24:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7354000000000003e-05 2020-11-16T20:24:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662194+0000", 2020-11-16T20:24:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020391761000000001 2020-11-16T20:24:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682585+0000", 2020-11-16T20:24:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114128 2020-11-16T20:24:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.682700+0000", 2020-11-16T20:24:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063062220000000002 2020-11-16T20:24:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689006+0000", 2020-11-16T20:24:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5025000000000007e-05 2020-11-16T20:24:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689091+0000", 2020-11-16T20:24:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024688079999999999 2020-11-16T20:24:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.691560+0000", 2020-11-16T20:24:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037529909 2020-11-16T20:24:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.954 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 31/26 ECSubWrite(tid=8, reqid=client.4444.0:110, at_version=31'4, trim_to=0'0, roll_forward_to=31'2))", 2020-11-16T20:24:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.175738651, 2020-11-16T20:24:28.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032919114999999999, 2020-11-16T20:24:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662056+0000", 2020-11-16T20:24:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6403999999999999e-05 2020-11-16T20:24:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662092+0000", 2020-11-16T20:24:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2906999999999999e-05 2020-11-16T20:24:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.662115+0000", 2020-11-16T20:24:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027791066999999999 2020-11-16T20:24:28.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689906+0000", 2020-11-16T20:24:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8580000000000002e-06 2020-11-16T20:24:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689913+0000", 2020-11-16T20:24:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0863999999999999e-05 2020-11-16T20:24:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689974+0000", 2020-11-16T20:24:28.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4901e-05 2020-11-16T20:24:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689989+0000", 2020-11-16T20:24:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049304020000000004 2020-11-16T20:24:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.694919+0000", 2020-11-16T20:24:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5711999999999998e-05 2020-11-16T20:24:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.694975+0000", 2020-11-16T20:24:28.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032919114999999999 2020-11-16T20:24:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s2 31/26 ECSubWrite(tid=10, reqid=client.4444.0:136, at_version=31'5, trim_to=0'0, roll_forward_to=31'3))", 2020-11-16T20:24:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:28.963 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.148462856, 2020-11-16T20:24:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042429041000000001, 2020-11-16T20:24:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689332+0000", 2020-11-16T20:24:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050619800000000002 2020-11-16T20:24:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689838+0000", 2020-11-16T20:24:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8632e-05 2020-11-16T20:24:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.689867+0000", 2020-11-16T20:24:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03374162 2020-11-16T20:24:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723608+0000", 2020-11-16T20:24:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5469999999999999e-06 2020-11-16T20:24:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723616+0000", 2020-11-16T20:24:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8924000000000006e-05 2020-11-16T20:24:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723685+0000", 2020-11-16T20:24:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7054e-05 2020-11-16T20:24:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.723702+0000", 2020-11-16T20:24:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080210799999999999 2020-11-16T20:24:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.731723+0000", 2020-11-16T20:24:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7985999999999998e-05 2020-11-16T20:24:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.731761+0000", 2020-11-16T20:24:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042429041000000001 2020-11-16T20:24:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.2s1 31/23 ECSubRead(tid=47, to_read={3:47d6be26:::smithi09216568-44:head=0,4096,0}, subchunks={3:47d6be26:::smithi09216568-44:head=[0,1]}, attrs_to_read=))", 2020-11-16T20:24:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.120948774, 2020-11-16T20:24:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027734008000000001, 2020-11-16T20:24:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716846+0000", 2020-11-16T20:24:28.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8687999999999998e-05 2020-11-16T20:24:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716905+0000", 2020-11-16T20:24:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0064999999999999e-05 2020-11-16T20:24:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.716915+0000", 2020-11-16T20:24:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026392019999999999 2020-11-16T20:24:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743307+0000", 2020-11-16T20:24:28.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1610999999999999e-05 2020-11-16T20:24:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743318+0000", 2020-11-16T20:24:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4503000000000001e-05 2020-11-16T20:24:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.743393+0000", 2020-11-16T20:24:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001187121 2020-11-16T20:24:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.744580+0000", 2020-11-16T20:24:28.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027734008000000001 2020-11-16T20:24:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:185 3.es0 3:746783df:::smithi09216568-20:head [append 0~507904] snapc 4=[4,3,2] ondisk+write+known_if_redirected e34)", 2020-11-16T20:24:28.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:18.576813+0000", 2020-11-16T20:24:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.260981765, 2020-11-16T20:24:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032237632000000002, 2020-11-16T20:24:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.979 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 185 2020-11-16T20:24:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.576813+0000", 2020-11-16T20:24:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.576813+0000", 2020-11-16T20:24:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036412800000000002 2020-11-16T20:24:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577177+0000", 2020-11-16T20:24:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0089999999999999e-05 2020-11-16T20:24:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577197+0000", 2020-11-16T20:24:28.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2817000000000001e-05 2020-11-16T20:24:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577230+0000", 2020-11-16T20:24:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7073000000000001e-05 2020-11-16T20:24:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577247+0000", 2020-11-16T20:24:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013625000000000001 2020-11-16T20:24:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577383+0000", 2020-11-16T20:24:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6784999999999999e-05 2020-11-16T20:24:28.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2020-11-16T20:24:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.577400+0000", 2020-11-16T20:24:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003451974 2020-11-16T20:24:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580852+0000", 2020-11-16T20:24:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012214800000000001 2020-11-16T20:24:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580974+0000", 2020-11-16T20:24:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001202928 2020-11-16T20:24:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.582177+0000", 2020-11-16T20:24:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018978376000000002 2020-11-16T20:24:28.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.601155+0000", 2020-11-16T20:24:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077913540000000003 2020-11-16T20:24:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:28.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.608947+0000", 2020-11-16T20:24:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010370900000000001 2020-11-16T20:24:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.609051+0000", 2020-11-16T20:24:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032237632000000002 2020-11-16T20:24:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:186 3.es0 3:746783df:::smithi09216568-20:head [append 0~401408] snapc 4=[4,3,2] ondisk+write+known_if_redirected e34)", 2020-11-16T20:24:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:18.579729+0000", 2020-11-16T20:24:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.258065267999999, 2020-11-16T20:24:28.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031962467000000001, 2020-11-16T20:24:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 186 2020-11-16T20:24:28.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.579729+0000", 2020-11-16T20:24:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.579729+0000", 2020-11-16T20:24:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062883899999999996 2020-11-16T20:24:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580358+0000", 2020-11-16T20:24:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8054999999999999e-05 2020-11-16T20:24:28.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580376+0000", 2020-11-16T20:24:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9548000000000002e-05 2020-11-16T20:24:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580416+0000", 2020-11-16T20:24:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4072e-05 2020-11-16T20:24:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:28.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580430+0000", 2020-11-16T20:24:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013003321999999999 2020-11-16T20:24:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.593433+0000", 2020-11-16T20:24:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021100700000000001 2020-11-16T20:24:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.593644+0000", 2020-11-16T20:24:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000748498 2020-11-16T20:24:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:28.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.594393+0000", 2020-11-16T20:24:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082229850000000004 2020-11-16T20:24:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.602616+0000", 2020-11-16T20:24:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089692920000000002 2020-11-16T20:24:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.611585+0000", 2020-11-16T20:24:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010684900000000001 2020-11-16T20:24:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:28.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.611692+0000", 2020-11-16T20:24:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031962467000000001 2020-11-16T20:24:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:187 3.es0 3:746783df:::smithi09216568-20:head [setxattr _header (56) in=7b] snapc 4=[4,3,2] ondisk+write+known_if_redirected e34)", 2020-11-16T20:24:29.000 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:18.580531+0000", 2020-11-16T20:24:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.257263668, 2020-11-16T20:24:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82514190300000001, 2020-11-16T20:24:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:29.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 187 2020-11-16T20:24:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580531+0000", 2020-11-16T20:24:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:29.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580531+0000", 2020-11-16T20:24:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133392 2020-11-16T20:24:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580664+0000", 2020-11-16T20:24:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6212e-05 2020-11-16T20:24:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580681+0000", 2020-11-16T20:24:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9286e-05 2020-11-16T20:24:29.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580710+0000", 2020-11-16T20:24:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4297000000000001e-05 2020-11-16T20:24:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580724+0000", 2020-11-16T20:24:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019066024000000001 2020-11-16T20:24:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.599790+0000", 2020-11-16T20:24:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112787 2020-11-16T20:24:29.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.599903+0000", 2020-11-16T20:24:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00065871699999999996 2020-11-16T20:24:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.600562+0000", 2020-11-16T20:24:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047374749999999997 2020-11-16T20:24:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.605299+0000", 2020-11-16T20:24:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80021618900000002 2020-11-16T20:24:29.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.405515+0000", 2020-11-16T20:24:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000157524 2020-11-16T20:24:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.405673+0000", 2020-11-16T20:24:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.82514190300000001 2020-11-16T20:24:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:188 3.es0 3:746783df:::smithi09216568-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e34)", 2020-11-16T20:24:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:18.580843+0000", 2020-11-16T20:24:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.256951540999999, 2020-11-16T20:24:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.828529878, 2020-11-16T20:24:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:29.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 188 2020-11-16T20:24:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580843+0000", 2020-11-16T20:24:29.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580843+0000", 2020-11-16T20:24:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1105000000000003e-05 2020-11-16T20:24:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580934+0000", 2020-11-16T20:24:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4115e-05 2020-11-16T20:24:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580948+0000", 2020-11-16T20:24:29.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8566999999999999e-05 2020-11-16T20:24:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580977+0000", 2020-11-16T20:24:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3980000000000005e-06 2020-11-16T20:24:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.580986+0000", 2020-11-16T20:24:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019924358 2020-11-16T20:24:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.600911+0000", 2020-11-16T20:24:29.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015903799999999999 2020-11-16T20:24:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:18.601070+0000", 2020-11-16T20:24:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80550670499999999 2020-11-16T20:24:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.406576+0000", 2020-11-16T20:24:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169784 2020-11-16T20:24:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.406746+0000", 2020-11-16T20:24:29.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002626808 2020-11-16T20:24:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:19.409373+0000", 2020-11-16T20:24:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.828529878 2020-11-16T20:24:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:29.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:302 3.5s0 3:aad7141b:::smithi09216568-18:head [copy-from ver 6] snapc 9=[] ondisk+write+known_if_redirected e41)", 2020-11-16T20:24:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:25.778620+0000", 2020-11-16T20:24:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.05917445, 2020-11-16T20:24:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034249157000000002, 2020-11-16T20:24:29.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 302 2020-11-16T20:24:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:29.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778620+0000", 2020-11-16T20:24:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:29.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778620+0000", 2020-11-16T20:24:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3764999999999996e-05 2020-11-16T20:24:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778694+0000", 2020-11-16T20:24:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4950000000000002e-06 2020-11-16T20:24:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778702+0000", 2020-11-16T20:24:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.467e-05 2020-11-16T20:24:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:29.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778716+0000", 2020-11-16T20:24:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3400000000000003e-06 2020-11-16T20:24:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778723+0000", 2020-11-16T20:24:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4474999999999994e-05 2020-11-16T20:24:29.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778797+0000", 2020-11-16T20:24:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014435399999999999 2020-11-16T20:24:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778941+0000", 2020-11-16T20:24:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023611064000000001 2020-11-16T20:24:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.802552+0000", 2020-11-16T20:24:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094922370000000006 2020-11-16T20:24:29.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.812045+0000", 2020-11-16T20:24:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011998999999999999 2020-11-16T20:24:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:29.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.812165+0000", 2020-11-16T20:24:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070476700000000002 2020-11-16T20:24:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.812869+0000", 2020-11-16T20:24:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034249157000000002 2020-11-16T20:24:29.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:303 3.5s0 3:aad7141b:::smithi09216568-18:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2020-11-16T20:24:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:25.778791+0000", 2020-11-16T20:24:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.0590036629999999, 2020-11-16T20:24:29.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034304929999999997, 2020-11-16T20:24:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 303 2020-11-16T20:24:29.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778791+0000", 2020-11-16T20:24:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778791+0000", 2020-11-16T20:24:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7741000000000003e-05 2020-11-16T20:24:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778839+0000", 2020-11-16T20:24:29.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.704e-06 2020-11-16T20:24:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778847+0000", 2020-11-16T20:24:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5947000000000002e-05 2020-11-16T20:24:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778863+0000", 2020-11-16T20:24:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.75e-06 2020-11-16T20:24:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.778869+0000", 2020-11-16T20:24:29.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025547999999999999 2020-11-16T20:24:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.779125+0000", 2020-11-16T20:24:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7084999999999998e-05 2020-11-16T20:24:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2020-11-16T20:24:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.779202+0000", 2020-11-16T20:24:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030106278 2020-11-16T20:24:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.809308+0000", 2020-11-16T20:24:29.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5284000000000005e-05 2020-11-16T20:24:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.809393+0000", 2020-11-16T20:24:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003511882 2020-11-16T20:24:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.812905+0000", 2020-11-16T20:24:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6650000000000006e-05 2020-11-16T20:24:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.812992+0000", 2020-11-16T20:24:29.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010412900000000001 2020-11-16T20:24:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:25.813096+0000", 2020-11-16T20:24:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034304929999999997 2020-11-16T20:24:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:29.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:29.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:29.041 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:29.044 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:29.044 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:29.044 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T20:24:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T20:24:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T20:24:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-11-16T20:24:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-11-16T20:24:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-11-16T20:24:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-11-16T20:24:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-11-16T20:24:29.046 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:24:29.046 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2020-11-16T20:24:28.654083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":25796608,"num_objects":60,"num_object_clones":10,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":149,"num_read_kb":12881,"num_write":288,"num_write_kb":27584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":85328,"kb_used_data":38608,"kb_used_omap":0,"kb_used_meta":46656,"kb_avail":754889392,"statfs":{"total":773094113280,"available":773006737408,"internally_reserved":0,"allocated":39534592,"data_stored":43237168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":47775744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020286"},"pg_stats":[{"pgid":"3.f","version":"31'2","reported_seq":"41","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.645095+0000","last_change":"2020-11-16T20:24:22.631089+0000","last_active":"2020-11-16T20:24:23.645095+0000","last_peered":"2020-11-16T20:24:23.645095+0000","last_clean":"2020-11-16T20:24:23.645095+0000","last_became_active":"2020-11-16T20:24:08.567854+0000","last_became_peered":"2020-11-16T20:24:08.567854+0000","last_unstale":"2020-11-16T20:24:23.645095+0000","last_undegraded":"2020-11-16T20:24:23.645095+0000","last_fullsized":"2020-11-16T20:24:23.645095+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":524288,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":1538,"num_write":2,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.e","version":"38'20","reported_seq":"57","reported_epoch":"38","state":"active+clean+snaptrim","last_fresh":"2020-11-16T20:24:23.615930+0000","last_change":"2020-11-16T20:24:23.609975+0000","last_active":"2020-11-16T20:24:23.615930+0000","last_peered":"2020-11-16T20:24:23.615930+0000","last_clean":"2020-11-16T20:24:23.615930+0000","last_became_active":"2020-11-16T20:24:08.572833+0000","last_became_peered":"2020-11-16T20:24:08.572833+0000","last_unstale":"2020-11-16T20:24:23.615930+0000","last_undegraded":"2020-11-16T20:24:23.615930+0000","last_fullsized":"2020-11-16T20:24:23.615930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":1589248,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":940,"num_write":29,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"3.d","version":"39'16","reported_seq":"55","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.682823+0000","last_change":"2020-11-16T20:24:23.605895+0000","last_active":"2020-11-16T20:24:23.682823+0000","last_peered":"2020-11-16T20:24:23.682823+0000","last_clean":"2020-11-16T20:24:23.682823+0000","last_became_active":"2020-11-16T20:24:08.567856+0000","last_became_peered":"2020-11-16T20:24:08.567856+0000","last_unstale":"2020-11-16T20:24:23.682823+0000","last_undegraded":"2020-11-16T20:24:23.682823+0000","last_fullsized":"2020-11-16T20:24:23.682823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":327680,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":603,"num_write":8,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.c","version":"31'10","reported_seq":"60","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.638788+0000","last_change":"2020-11-16T20:24:22.631409+0000","last_active":"2020-11-16T20:24:23.638788+0000","last_peered":"2020-11-16T20:24:23.638788+0000","last_clean":"2020-11-16T20:24:23.638788+0000","last_became_active":"2020-11-16T20:24:08.560519+0000","last_became_peered":"2020-11-16T20:24:08.560519+0000","last_unstale":"2020-11-16T20:24:23.638788+0000","last_undegraded":"2020-11-16T20:24:23.638788+0000","last_fullsized":"2020-11-16T20:24:23.638788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2146304,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":4736,"num_write":10,"num_write_kb":2096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.9","version":"39'31","reported_seq":"70","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.665480+0000","last_change":"2020-11-16T20:24:23.607220+0000","last_active":"2020-11-16T20:24:23.665480+0000","last_peered":"2020-11-16T20:24:23.665480+0000","last_clean":"2020-11-16T20:24:23.665480+0000","last_became_active":"2020-11-16T20:24:08.567696+0000","last_became_peered":"2020-11-16T20:24:08.567696+0000","last_unstale":"2020-11-16T20:24:23.665480+0000","last_undegraded":"2020-11-16T20:24:23.665480+0000","last_fullsized":"2020-11-16T20:24:23.665480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2400256,"num_objects":7,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":6,"num_write":38,"num_write_kb":2512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.8","version":"38'16","reported_seq":"54","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:23.613133+0000","last_change":"2020-11-16T20:24:23.613133+0000","last_active":"2020-11-16T20:24:23.613133+0000","last_peered":"2020-11-16T20:24:23.613133+0000","last_clean":"2020-11-16T20:24:23.613133+0000","last_became_active":"2020-11-16T20:24:08.560349+0000","last_became_peered":"2020-11-16T20:24:08.560349+0000","last_unstale":"2020-11-16T20:24:23.613133+0000","last_undegraded":"2020-11-16T20:24:23.613133+0000","last_fullsized":"2020-11-16T20:24:23.613133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1425408,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":30,"num_write_kb":1392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"1.a","version":"14'2","reported_seq":"61","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.610482+0000","last_change":"2020-11-16T20:24:18.638502+0000","last_active":"2020-11-16T20:24:22.610482+0000","last_peered":"2020-11-16T20:24:22.610482+0000","last_clean":"2020-11-16T20:24:22.610482+0000","last_became_active":"2020-11-16T20:24:14.530383+0000","last_became_peered":"2020-11-16T20:24:14.530383+0000","last_unstale":"2020-11-16T20:24:22.610482+0000","last_undegraded":"2020-11-16T20:24:22.610482+0000","last_fullsized":"2020-11-16T20:24:22.610482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:18.638373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:18.638373+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"38'8","reported_seq":"45","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.636932+0000","last_change":"2020-11-16T20:24:22.618198+0000","last_active":"2020-11-16T20:24:22.636932+0000","last_peered":"2020-11-16T20:24:22.636932+0000","last_clean":"2020-11-16T20:24:22.636932+0000","last_became_active":"2020-11-16T20:24:08.561017+0000","last_became_peered":"2020-11-16T20:24:08.561017+0000","last_unstale":"2020-11-16T20:24:22.636932+0000","last_undegraded":"2020-11-16T20:24:22.636932+0000","last_fullsized":"2020-11-16T20:24:22.636932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":917504,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":6,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"1.9","version":"14'2","reported_seq":"65","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.610497+0000","last_change":"2020-11-16T20:24:14.649226+0000","last_active":"2020-11-16T20:24:22.610497+0000","last_peered":"2020-11-16T20:24:22.610497+0000","last_clean":"2020-11-16T20:24:22.610497+0000","last_became_active":"2020-11-16T20:24:14.529598+0000","last_became_peered":"2020-11-16T20:24:14.529598+0000","last_unstale":"2020-11-16T20:24:22.610497+0000","last_undegraded":"2020-11-16T20:24:22.610497+0000","last_fullsized":"2020-11-16T20:24:22.610497+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:14.649047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:14.649047+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"31'2","reported_seq":"39","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:23.604735+0000","last_change":"2020-11-16T20:24:23.604735+0000","last_active":"2020-11-16T20:24:23.604735+0000","last_peered":"2020-11-16T20:24:23.604735+0000","last_clean":"2020-11-16T20:24:23.604735+0000","last_became_active":"2020-11-16T20:24:08.564124+0000","last_became_peered":"2020-11-16T20:24:08.564124+0000","last_unstale":"2020-11-16T20:24:23.604735+0000","last_undegraded":"2020-11-16T20:24:23.604735+0000","last_fullsized":"2020-11-16T20:24:23.604735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"1.8","version":"14'2","reported_seq":"61","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.610580+0000","last_change":"2020-11-16T20:24:15.680985+0000","last_active":"2020-11-16T20:24:22.610580+0000","last_peered":"2020-11-16T20:24:22.610580+0000","last_clean":"2020-11-16T20:24:22.610580+0000","last_became_active":"2020-11-16T20:24:14.530096+0000","last_became_peered":"2020-11-16T20:24:14.530096+0000","last_unstale":"2020-11-16T20:24:22.610580+0000","last_undegraded":"2020-11-16T20:24:22.610580+0000","last_fullsized":"2020-11-16T20:24:22.610580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:15.680854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:15.680854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"39'18","reported_seq":"55","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.661461+0000","last_change":"2020-11-16T20:24:23.610523+0000","last_active":"2020-11-16T20:24:23.661461+0000","last_peered":"2020-11-16T20:24:23.661461+0000","last_clean":"2020-11-16T20:24:23.661461+0000","last_became_active":"2020-11-16T20:24:08.564324+0000","last_became_peered":"2020-11-16T20:24:08.564324+0000","last_unstale":"2020-11-16T20:24:23.661461+0000","last_undegraded":"2020-11-16T20:24:23.661461+0000","last_fullsized":"2020-11-16T20:24:23.661461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3383296,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":3304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":"23","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.606366+0000","last_change":"2020-11-16T20:24:11.491153+0000","last_active":"2020-11-16T20:24:22.606366+0000","last_peered":"2020-11-16T20:24:22.606366+0000","last_clean":"2020-11-16T20:24:22.606366+0000","last_became_active":"2020-11-16T20:24:11.490769+0000","last_became_peered":"2020-11-16T20:24:11.490769+0000","last_unstale":"2020-11-16T20:24:22.606366+0000","last_undegraded":"2020-11-16T20:24:22.606366+0000","last_fullsized":"2020-11-16T20:24:22.606366+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"14'2","reported_seq":"58","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.611530+0000","last_change":"2020-11-16T20:24:14.531583+0000","last_active":"2020-11-16T20:24:22.611530+0000","last_peered":"2020-11-16T20:24:22.611530+0000","last_clean":"2020-11-16T20:24:22.611530+0000","last_became_active":"2020-11-16T20:24:14.531025+0000","last_became_peered":"2020-11-16T20:24:14.531025+0000","last_unstale":"2020-11-16T20:24:22.611530+0000","last_undegraded":"2020-11-16T20:24:22.611530+0000","last_fullsized":"2020-11-16T20:24:22.611530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:23:36.602638+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"31'4","reported_seq":"41","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.630522+0000","last_change":"2020-11-16T20:24:22.630522+0000","last_active":"2020-11-16T20:24:22.630522+0000","last_peered":"2020-11-16T20:24:22.630522+0000","last_clean":"2020-11-16T20:24:22.630522+0000","last_became_active":"2020-11-16T20:24:08.566542+0000","last_became_peered":"2020-11-16T20:24:08.566542+0000","last_unstale":"2020-11-16T20:24:22.630522+0000","last_undegraded":"2020-11-16T20:24:22.630522+0000","last_fullsized":"2020-11-16T20:24:22.630522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1032192,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":1008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.2","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624499+0000","last_change":"2020-11-16T20:23:44.606663+0000","last_active":"2020-11-16T20:24:22.624499+0000","last_peered":"2020-11-16T20:24:22.624499+0000","last_clean":"2020-11-16T20:24:22.624499+0000","last_became_active":"2020-11-16T20:23:44.605904+0000","last_became_peered":"2020-11-16T20:23:44.605904+0000","last_unstale":"2020-11-16T20:24:22.624499+0000","last_undegraded":"2020-11-16T20:24:22.624499+0000","last_fullsized":"2020-11-16T20:24:22.624499+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"14'2","reported_seq":"31","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624082+0000","last_change":"2020-11-16T20:24:16.301062+0000","last_active":"2020-11-16T20:24:22.624082+0000","last_peered":"2020-11-16T20:24:22.624082+0000","last_clean":"2020-11-16T20:24:22.624082+0000","last_became_active":"2020-11-16T20:24:14.531581+0000","last_became_peered":"2020-11-16T20:24:14.531581+0000","last_unstale":"2020-11-16T20:24:22.624082+0000","last_undegraded":"2020-11-16T20:24:22.624082+0000","last_fullsized":"2020-11-16T20:24:22.624082+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:16.300952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:16.300952+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"38'13","reported_seq":"55","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:23.229377+0000","last_change":"2020-11-16T20:24:22.640375+0000","last_active":"2020-11-16T20:24:23.229377+0000","last_peered":"2020-11-16T20:24:23.229377+0000","last_clean":"2020-11-16T20:24:23.229377+0000","last_became_active":"2020-11-16T20:24:11.494794+0000","last_became_peered":"2020-11-16T20:24:11.494794+0000","last_unstale":"2020-11-16T20:24:23.229377+0000","last_undegraded":"2020-11-16T20:24:23.229377+0000","last_fullsized":"2020-11-16T20:24:23.229377+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1744896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":2237,"num_write":10,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.617588+0000","last_change":"2020-11-16T20:23:44.493249+0000","last_active":"2020-11-16T20:24:22.617588+0000","last_peered":"2020-11-16T20:24:22.617588+0000","last_clean":"2020-11-16T20:24:22.617588+0000","last_became_active":"2020-11-16T20:23:44.492834+0000","last_became_peered":"2020-11-16T20:23:44.492834+0000","last_unstale":"2020-11-16T20:24:22.617588+0000","last_undegraded":"2020-11-16T20:24:22.617588+0000","last_fullsized":"2020-11-16T20:24:22.617588+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.621039+0000","last_change":"2020-11-16T20:24:14.608610+0000","last_active":"2020-11-16T20:24:22.621039+0000","last_peered":"2020-11-16T20:24:22.621039+0000","last_clean":"2020-11-16T20:24:22.621039+0000","last_became_active":"2020-11-16T20:24:14.527213+0000","last_became_peered":"2020-11-16T20:24:14.527213+0000","last_unstale":"2020-11-16T20:24:22.621039+0000","last_undegraded":"2020-11-16T20:24:22.621039+0000","last_fullsized":"2020-11-16T20:24:22.621039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:14.608430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:14.608430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"38'20","reported_seq":"50","reported_epoch":"38","state":"active+clean+snaptrim","last_fresh":"2020-11-16T20:24:23.615993+0000","last_change":"2020-11-16T20:24:23.605426+0000","last_active":"2020-11-16T20:24:23.615993+0000","last_peered":"2020-11-16T20:24:23.615993+0000","last_clean":"2020-11-16T20:24:23.615993+0000","last_became_active":"2020-11-16T20:24:08.563031+0000","last_became_peered":"2020-11-16T20:24:08.563031+0000","last_unstale":"2020-11-16T20:24:23.615993+0000","last_undegraded":"2020-11-16T20:24:23.615993+0000","last_fullsized":"2020-11-16T20:24:23.615993+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":2424832,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":29,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624573+0000","last_change":"2020-11-16T20:23:44.501339+0000","last_active":"2020-11-16T20:24:22.624573+0000","last_peered":"2020-11-16T20:24:22.624573+0000","last_clean":"2020-11-16T20:24:22.624573+0000","last_became_active":"2020-11-16T20:23:44.500782+0000","last_became_peered":"2020-11-16T20:23:44.500782+0000","last_unstale":"2020-11-16T20:24:22.624573+0000","last_undegraded":"2020-11-16T20:24:22.624573+0000","last_fullsized":"2020-11-16T20:24:22.624573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"47","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.610872+0000","last_change":"2020-11-16T20:24:11.657961+0000","last_active":"2020-11-16T20:24:22.610872+0000","last_peered":"2020-11-16T20:24:22.610872+0000","last_clean":"2020-11-16T20:24:22.610872+0000","last_became_active":"2020-11-16T20:24:07.559654+0000","last_became_peered":"2020-11-16T20:24:07.559654+0000","last_unstale":"2020-11-16T20:24:22.610872+0000","last_undegraded":"2020-11-16T20:24:22.610872+0000","last_fullsized":"2020-11-16T20:24:22.610872+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:11.657774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:11.657774+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"38'30","reported_seq":"74","reported_epoch":"39","state":"active+clean+snaptrim","last_fresh":"2020-11-16T20:24:23.637172+0000","last_change":"2020-11-16T20:24:23.612594+0000","last_active":"2020-11-16T20:24:23.637172+0000","last_peered":"2020-11-16T20:24:23.637172+0000","last_clean":"2020-11-16T20:24:23.637172+0000","last_became_active":"2020-11-16T20:24:08.563332+0000","last_became_peered":"2020-11-16T20:24:08.563332+0000","last_unstale":"2020-11-16T20:24:23.637172+0000","last_undegraded":"2020-11-16T20:24:23.637172+0000","last_fullsized":"2020-11-16T20:24:23.637172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":2277376,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":2792,"num_write":15,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624607+0000","last_change":"2020-11-16T20:23:44.503055+0000","last_active":"2020-11-16T20:24:22.624607+0000","last_peered":"2020-11-16T20:24:22.624607+0000","last_clean":"2020-11-16T20:24:22.624607+0000","last_became_active":"2020-11-16T20:23:44.502588+0000","last_became_peered":"2020-11-16T20:23:44.502588+0000","last_unstale":"2020-11-16T20:24:22.624607+0000","last_undegraded":"2020-11-16T20:24:22.624607+0000","last_fullsized":"2020-11-16T20:24:22.624607+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":"54","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.612357+0000","last_change":"2020-11-16T20:24:15.472709+0000","last_active":"2020-11-16T20:24:22.612357+0000","last_peered":"2020-11-16T20:24:22.612357+0000","last_clean":"2020-11-16T20:24:22.612357+0000","last_became_active":"2020-11-16T20:24:14.526832+0000","last_became_peered":"2020-11-16T20:24:14.526832+0000","last_unstale":"2020-11-16T20:24:22.612357+0000","last_undegraded":"2020-11-16T20:24:22.612357+0000","last_fullsized":"2020-11-16T20:24:22.612357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:15.472519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:15.472519+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"39'13","reported_seq":"53","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.625129+0000","last_change":"2020-11-16T20:24:22.606898+0000","last_active":"2020-11-16T20:24:23.625129+0000","last_peered":"2020-11-16T20:24:23.625129+0000","last_clean":"2020-11-16T20:24:23.625129+0000","last_became_active":"2020-11-16T20:24:08.558421+0000","last_became_peered":"2020-11-16T20:24:08.558421+0000","last_unstale":"2020-11-16T20:24:23.625129+0000","last_undegraded":"2020-11-16T20:24:23.625129+0000","last_fullsized":"2020-11-16T20:24:23.625129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2244608,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":30,"num_write_kb":2192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.625384+0000","last_change":"2020-11-16T20:23:44.501820+0000","last_active":"2020-11-16T20:24:22.625384+0000","last_peered":"2020-11-16T20:24:22.625384+0000","last_clean":"2020-11-16T20:24:22.625384+0000","last_became_active":"2020-11-16T20:23:44.501424+0000","last_became_peered":"2020-11-16T20:23:44.501424+0000","last_unstale":"2020-11-16T20:24:22.625384+0000","last_undegraded":"2020-11-16T20:24:22.625384+0000","last_fullsized":"2020-11-16T20:24:22.625384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"14'2","reported_seq":"31","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.625012+0000","last_change":"2020-11-16T20:24:15.309347+0000","last_active":"2020-11-16T20:24:22.625012+0000","last_peered":"2020-11-16T20:24:22.625012+0000","last_clean":"2020-11-16T20:24:22.625012+0000","last_became_active":"2020-11-16T20:24:14.530884+0000","last_became_peered":"2020-11-16T20:24:14.530884+0000","last_unstale":"2020-11-16T20:24:22.625012+0000","last_undegraded":"2020-11-16T20:24:22.625012+0000","last_fullsized":"2020-11-16T20:24:22.625012+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:15.309247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:15.309247+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"38'23","reported_seq":"58","reported_epoch":"38","state":"active+clean+snaptrim","last_fresh":"2020-11-16T20:24:23.620532+0000","last_change":"2020-11-16T20:24:23.615618+0000","last_active":"2020-11-16T20:24:23.620532+0000","last_peered":"2020-11-16T20:24:23.620532+0000","last_clean":"2020-11-16T20:24:23.620532+0000","last_became_active":"2020-11-16T20:24:08.559749+0000","last_became_peered":"2020-11-16T20:24:08.559749+0000","last_unstale":"2020-11-16T20:24:23.620532+0000","last_undegraded":"2020-11-16T20:24:23.620532+0000","last_fullsized":"2020-11-16T20:24:23.620532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":1818624,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":5,"num_write":33,"num_write_kb":2784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":"29","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.614067+0000","last_change":"2020-11-16T20:23:44.598654+0000","last_active":"2020-11-16T20:24:22.614067+0000","last_peered":"2020-11-16T20:24:22.614067+0000","last_clean":"2020-11-16T20:24:22.614067+0000","last_became_active":"2020-11-16T20:23:44.597867+0000","last_became_peered":"2020-11-16T20:23:44.597867+0000","last_unstale":"2020-11-16T20:24:22.614067+0000","last_undegraded":"2020-11-16T20:24:22.614067+0000","last_fullsized":"2020-11-16T20:24:22.614067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"14'2","reported_seq":"31","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.614354+0000","last_change":"2020-11-16T20:24:19.329617+0000","last_active":"2020-11-16T20:24:22.614354+0000","last_peered":"2020-11-16T20:24:22.614354+0000","last_clean":"2020-11-16T20:24:22.614354+0000","last_became_active":"2020-11-16T20:24:14.529112+0000","last_became_peered":"2020-11-16T20:24:14.529112+0000","last_unstale":"2020-11-16T20:24:22.614354+0000","last_undegraded":"2020-11-16T20:24:22.614354+0000","last_fullsized":"2020-11-16T20:24:22.614354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:19.329415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:19.329415+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"39'10","reported_seq":"45","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.621985+0000","last_change":"2020-11-16T20:24:23.615500+0000","last_active":"2020-11-16T20:24:23.621985+0000","last_peered":"2020-11-16T20:24:23.621985+0000","last_clean":"2020-11-16T20:24:23.621985+0000","last_became_active":"2020-11-16T20:24:08.553683+0000","last_became_peered":"2020-11-16T20:24:08.553683+0000","last_unstale":"2020-11-16T20:24:23.621985+0000","last_undegraded":"2020-11-16T20:24:23.621985+0000","last_fullsized":"2020-11-16T20:24:23.621985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":950272,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":30,"num_write_kb":928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624443+0000","last_change":"2020-11-16T20:23:44.605343+0000","last_active":"2020-11-16T20:24:22.624443+0000","last_peered":"2020-11-16T20:24:22.624443+0000","last_clean":"2020-11-16T20:24:22.624443+0000","last_became_active":"2020-11-16T20:23:44.604919+0000","last_became_peered":"2020-11-16T20:23:44.604919+0000","last_unstale":"2020-11-16T20:24:22.624443+0000","last_undegraded":"2020-11-16T20:24:22.624443+0000","last_fullsized":"2020-11-16T20:24:22.624443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"14'2","reported_seq":"25","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.617196+0000","last_change":"2020-11-16T20:24:16.437595+0000","last_active":"2020-11-16T20:24:22.617196+0000","last_peered":"2020-11-16T20:24:22.617196+0000","last_clean":"2020-11-16T20:24:22.617196+0000","last_became_active":"2020-11-16T20:24:14.527077+0000","last_became_peered":"2020-11-16T20:24:14.527077+0000","last_unstale":"2020-11-16T20:24:22.617196+0000","last_undegraded":"2020-11-16T20:24:22.617196+0000","last_fullsized":"2020-11-16T20:24:22.617196+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:16.437407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:16.437407+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":25796608,"num_objects":58,"num_object_clones":10,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":149,"num_read_kb":12881,"num_write":288,"num_write_kb":27584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38694912,"data_stored":45563904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":22,"acting":22,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9532,"kb_used_data":4212,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94362308,"statfs":{"total":96636764160,"available":96627003392,"internally_reserved":0,"allocated":4313088,"data_stored":4615654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.835}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4220000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1960000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0619999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.913}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2909999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10648,"kb_used_data":5264,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94361192,"statfs":{"total":96636764160,"available":96625860608,"internally_reserved":0,"allocated":5390336,"data_stored":5668326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4849999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9390000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.786}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5670000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.714}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7920,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94363920,"statfs":{"total":96636764160,"available":96628654080,"internally_reserved":0,"allocated":2531328,"data_stored":2420198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6509999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11232,"kb_used_data":4632,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":94360608,"statfs":{"total":96636764160,"available":96625262592,"internally_reserved":0,"allocated":4743168,"data_stored":5508582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1640000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3279999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6120000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2879999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6749999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14192,"kb_used_data":7528,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94357648,"statfs":{"total":96636764160,"available":96622231552,"internally_reserved":0,"allocated":7708672,"data_stored":9117158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4750000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7650000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.577}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.609}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5489999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7724,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94364116,"statfs":{"total":96636764160,"available":96628854784,"internally_reserved":0,"allocated":2658304,"data_stored":2420198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2559999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3439999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.399}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14532,"kb_used_data":8764,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94357308,"statfs":{"total":96636764160,"available":96621883392,"internally_reserved":0,"allocated":8974336,"data_stored":9735654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.796}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4390000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3239999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6549999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9548,"kb_used_data":3140,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94362292,"statfs":{"total":96636764160,"available":96626987008,"internally_reserved":0,"allocated":3215360,"data_stored":3751398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1909999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1389999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.669}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6230000000000002}]}]}]}} 2020-11-16T20:24:29.047 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2020-11-16T20:24:29.050 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-16T20:24:29.050 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:29.041+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:29.051 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-11-16T20:24:29.143 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:29.141+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:29.208 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:29.208 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:29.209 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2020-11-16T20:24:29.209 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:29.220 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-11-16T20:24:29.264 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:29.265+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:29.344 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:29.340+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2020-11-16T20:24:29.382 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:29.383 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:29.384 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2020-11-16T20:24:29.392 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-11-16T20:24:29.397 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2020-11-16T20:24:28.654083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":25796608,"num_objects":60,"num_object_clones":10,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":149,"num_read_kb":12881,"num_write":288,"num_write_kb":27584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":85328,"kb_used_data":38608,"kb_used_omap":0,"kb_used_meta":46656,"kb_avail":754889392,"statfs":{"total":773094113280,"available":773006737408,"internally_reserved":0,"allocated":39534592,"data_stored":43237168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":47775744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020286"},"pg_stats":[{"pgid":"3.f","version":"31'2","reported_seq":"41","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.645095+0000","last_change":"2020-11-16T20:24:22.631089+0000","last_active":"2020-11-16T20:24:23.645095+0000","last_peered":"2020-11-16T20:24:23.645095+0000","last_clean":"2020-11-16T20:24:23.645095+0000","last_became_active":"2020-11-16T20:24:08.567854+0000","last_became_peered":"2020-11-16T20:24:08.567854+0000","last_unstale":"2020-11-16T20:24:23.645095+0000","last_undegraded":"2020-11-16T20:24:23.645095+0000","last_fullsized":"2020-11-16T20:24:23.645095+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":524288,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":1538,"num_write":2,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.e","version":"38'20","reported_seq":"57","reported_epoch":"38","state":"active+clean+snaptrim","last_fresh":"2020-11-16T20:24:23.615930+0000","last_change":"2020-11-16T20:24:23.609975+0000","last_active":"2020-11-16T20:24:23.615930+0000","last_peered":"2020-11-16T20:24:23.615930+0000","last_clean":"2020-11-16T20:24:23.615930+0000","last_became_active":"2020-11-16T20:24:08.572833+0000","last_became_peered":"2020-11-16T20:24:08.572833+0000","last_unstale":"2020-11-16T20:24:23.615930+0000","last_undegraded":"2020-11-16T20:24:23.615930+0000","last_fullsized":"2020-11-16T20:24:23.615930+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":1589248,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":940,"num_write":29,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"3.d","version":"39'16","reported_seq":"55","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.682823+0000","last_change":"2020-11-16T20:24:23.605895+0000","last_active":"2020-11-16T20:24:23.682823+0000","last_peered":"2020-11-16T20:24:23.682823+0000","last_clean":"2020-11-16T20:24:23.682823+0000","last_became_active":"2020-11-16T20:24:08.567856+0000","last_became_peered":"2020-11-16T20:24:08.567856+0000","last_unstale":"2020-11-16T20:24:23.682823+0000","last_undegraded":"2020-11-16T20:24:23.682823+0000","last_fullsized":"2020-11-16T20:24:23.682823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":327680,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":603,"num_write":8,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.c","version":"31'10","reported_seq":"60","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.638788+0000","last_change":"2020-11-16T20:24:22.631409+0000","last_active":"2020-11-16T20:24:23.638788+0000","last_peered":"2020-11-16T20:24:23.638788+0000","last_clean":"2020-11-16T20:24:23.638788+0000","last_became_active":"2020-11-16T20:24:08.560519+0000","last_became_peered":"2020-11-16T20:24:08.560519+0000","last_unstale":"2020-11-16T20:24:23.638788+0000","last_undegraded":"2020-11-16T20:24:23.638788+0000","last_fullsized":"2020-11-16T20:24:23.638788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2146304,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":4736,"num_write":10,"num_write_kb":2096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.9","version":"39'31","reported_seq":"70","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.665480+0000","last_change":"2020-11-16T20:24:23.607220+0000","last_active":"2020-11-16T20:24:23.665480+0000","last_peered":"2020-11-16T20:24:23.665480+0000","last_clean":"2020-11-16T20:24:23.665480+0000","last_became_active":"2020-11-16T20:24:08.567696+0000","last_became_peered":"2020-11-16T20:24:08.567696+0000","last_unstale":"2020-11-16T20:24:23.665480+0000","last_undegraded":"2020-11-16T20:24:23.665480+0000","last_fullsized":"2020-11-16T20:24:23.665480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2400256,"num_objects":7,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":12,"num_read_kb":6,"num_write":38,"num_write_kb":2512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"3.8","version":"38'16","reported_seq":"54","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:23.613133+0000","last_change":"2020-11-16T20:24:23.613133+0000","last_active":"2020-11-16T20:24:23.613133+0000","last_peered":"2020-11-16T20:24:23.613133+0000","last_clean":"2020-11-16T20:24:23.613133+0000","last_became_active":"2020-11-16T20:24:08.560349+0000","last_became_peered":"2020-11-16T20:24:08.560349+0000","last_unstale":"2020-11-16T20:24:23.613133+0000","last_undegraded":"2020-11-16T20:24:23.613133+0000","last_fullsized":"2020-11-16T20:24:23.613133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1425408,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":30,"num_write_kb":1392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"1.a","version":"14'2","reported_seq":"61","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.610482+0000","last_change":"2020-11-16T20:24:18.638502+0000","last_active":"2020-11-16T20:24:22.610482+0000","last_peered":"2020-11-16T20:24:22.610482+0000","last_clean":"2020-11-16T20:24:22.610482+0000","last_became_active":"2020-11-16T20:24:14.530383+0000","last_became_peered":"2020-11-16T20:24:14.530383+0000","last_unstale":"2020-11-16T20:24:22.610482+0000","last_undegraded":"2020-11-16T20:24:22.610482+0000","last_fullsized":"2020-11-16T20:24:22.610482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:18.638373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:18.638373+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"38'8","reported_seq":"45","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.636932+0000","last_change":"2020-11-16T20:24:22.618198+0000","last_active":"2020-11-16T20:24:22.636932+0000","last_peered":"2020-11-16T20:24:22.636932+0000","last_clean":"2020-11-16T20:24:22.636932+0000","last_became_active":"2020-11-16T20:24:08.561017+0000","last_became_peered":"2020-11-16T20:24:08.561017+0000","last_unstale":"2020-11-16T20:24:22.636932+0000","last_undegraded":"2020-11-16T20:24:22.636932+0000","last_fullsized":"2020-11-16T20:24:22.636932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":917504,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":6,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"1.9","version":"14'2","reported_seq":"65","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.610497+0000","last_change":"2020-11-16T20:24:14.649226+0000","last_active":"2020-11-16T20:24:22.610497+0000","last_peered":"2020-11-16T20:24:22.610497+0000","last_clean":"2020-11-16T20:24:22.610497+0000","last_became_active":"2020-11-16T20:24:14.529598+0000","last_became_peered":"2020-11-16T20:24:14.529598+0000","last_unstale":"2020-11-16T20:24:22.610497+0000","last_undegraded":"2020-11-16T20:24:22.610497+0000","last_fullsized":"2020-11-16T20:24:22.610497+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:14.649047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:14.649047+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"31'2","reported_seq":"39","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:23.604735+0000","last_change":"2020-11-16T20:24:23.604735+0000","last_active":"2020-11-16T20:24:23.604735+0000","last_peered":"2020-11-16T20:24:23.604735+0000","last_clean":"2020-11-16T20:24:23.604735+0000","last_became_active":"2020-11-16T20:24:08.564124+0000","last_became_peered":"2020-11-16T20:24:08.564124+0000","last_unstale":"2020-11-16T20:24:23.604735+0000","last_undegraded":"2020-11-16T20:24:23.604735+0000","last_fullsized":"2020-11-16T20:24:23.604735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":589824,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"1.8","version":"14'2","reported_seq":"61","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.610580+0000","last_change":"2020-11-16T20:24:15.680985+0000","last_active":"2020-11-16T20:24:22.610580+0000","last_peered":"2020-11-16T20:24:22.610580+0000","last_clean":"2020-11-16T20:24:22.610580+0000","last_became_active":"2020-11-16T20:24:14.530096+0000","last_became_peered":"2020-11-16T20:24:14.530096+0000","last_unstale":"2020-11-16T20:24:22.610580+0000","last_undegraded":"2020-11-16T20:24:22.610580+0000","last_fullsized":"2020-11-16T20:24:22.610580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:15.680854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:15.680854+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"39'18","reported_seq":"55","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.661461+0000","last_change":"2020-11-16T20:24:23.610523+0000","last_active":"2020-11-16T20:24:23.661461+0000","last_peered":"2020-11-16T20:24:23.661461+0000","last_clean":"2020-11-16T20:24:23.661461+0000","last_became_active":"2020-11-16T20:24:08.564324+0000","last_became_peered":"2020-11-16T20:24:08.564324+0000","last_unstale":"2020-11-16T20:24:23.661461+0000","last_undegraded":"2020-11-16T20:24:23.661461+0000","last_fullsized":"2020-11-16T20:24:23.661461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3383296,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":3304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":"23","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.606366+0000","last_change":"2020-11-16T20:24:11.491153+0000","last_active":"2020-11-16T20:24:22.606366+0000","last_peered":"2020-11-16T20:24:22.606366+0000","last_clean":"2020-11-16T20:24:22.606366+0000","last_became_active":"2020-11-16T20:24:11.490769+0000","last_became_peered":"2020-11-16T20:24:11.490769+0000","last_unstale":"2020-11-16T20:24:22.606366+0000","last_undegraded":"2020-11-16T20:24:22.606366+0000","last_fullsized":"2020-11-16T20:24:22.606366+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"14'2","reported_seq":"58","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.611530+0000","last_change":"2020-11-16T20:24:14.531583+0000","last_active":"2020-11-16T20:24:22.611530+0000","last_peered":"2020-11-16T20:24:22.611530+0000","last_clean":"2020-11-16T20:24:22.611530+0000","last_became_active":"2020-11-16T20:24:14.531025+0000","last_became_peered":"2020-11-16T20:24:14.531025+0000","last_unstale":"2020-11-16T20:24:22.611530+0000","last_undegraded":"2020-11-16T20:24:22.611530+0000","last_fullsized":"2020-11-16T20:24:22.611530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:23:36.602638+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"31'4","reported_seq":"41","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.630522+0000","last_change":"2020-11-16T20:24:22.630522+0000","last_active":"2020-11-16T20:24:22.630522+0000","last_peered":"2020-11-16T20:24:22.630522+0000","last_clean":"2020-11-16T20:24:22.630522+0000","last_became_active":"2020-11-16T20:24:08.566542+0000","last_became_peered":"2020-11-16T20:24:08.566542+0000","last_unstale":"2020-11-16T20:24:22.630522+0000","last_undegraded":"2020-11-16T20:24:22.630522+0000","last_fullsized":"2020-11-16T20:24:22.630522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1032192,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":1008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.2","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624499+0000","last_change":"2020-11-16T20:23:44.606663+0000","last_active":"2020-11-16T20:24:22.624499+0000","last_peered":"2020-11-16T20:24:22.624499+0000","last_clean":"2020-11-16T20:24:22.624499+0000","last_became_active":"2020-11-16T20:23:44.605904+0000","last_became_peered":"2020-11-16T20:23:44.605904+0000","last_unstale":"2020-11-16T20:24:22.624499+0000","last_undegraded":"2020-11-16T20:24:22.624499+0000","last_fullsized":"2020-11-16T20:24:22.624499+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"14'2","reported_seq":"31","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624082+0000","last_change":"2020-11-16T20:24:16.301062+0000","last_active":"2020-11-16T20:24:22.624082+0000","last_peered":"2020-11-16T20:24:22.624082+0000","last_clean":"2020-11-16T20:24:22.624082+0000","last_became_active":"2020-11-16T20:24:14.531581+0000","last_became_peered":"2020-11-16T20:24:14.531581+0000","last_unstale":"2020-11-16T20:24:22.624082+0000","last_undegraded":"2020-11-16T20:24:22.624082+0000","last_fullsized":"2020-11-16T20:24:22.624082+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:16.300952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:16.300952+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"38'13","reported_seq":"55","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:23.229377+0000","last_change":"2020-11-16T20:24:22.640375+0000","last_active":"2020-11-16T20:24:23.229377+0000","last_peered":"2020-11-16T20:24:23.229377+0000","last_clean":"2020-11-16T20:24:23.229377+0000","last_became_active":"2020-11-16T20:24:11.494794+0000","last_became_peered":"2020-11-16T20:24:11.494794+0000","last_unstale":"2020-11-16T20:24:23.229377+0000","last_undegraded":"2020-11-16T20:24:23.229377+0000","last_fullsized":"2020-11-16T20:24:23.229377+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1744896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":2237,"num_write":10,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.617588+0000","last_change":"2020-11-16T20:23:44.493249+0000","last_active":"2020-11-16T20:24:22.617588+0000","last_peered":"2020-11-16T20:24:22.617588+0000","last_clean":"2020-11-16T20:24:22.617588+0000","last_became_active":"2020-11-16T20:23:44.492834+0000","last_became_peered":"2020-11-16T20:23:44.492834+0000","last_unstale":"2020-11-16T20:24:22.617588+0000","last_undegraded":"2020-11-16T20:24:22.617588+0000","last_fullsized":"2020-11-16T20:24:22.617588+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.621039+0000","last_change":"2020-11-16T20:24:14.608610+0000","last_active":"2020-11-16T20:24:22.621039+0000","last_peered":"2020-11-16T20:24:22.621039+0000","last_clean":"2020-11-16T20:24:22.621039+0000","last_became_active":"2020-11-16T20:24:14.527213+0000","last_became_peered":"2020-11-16T20:24:14.527213+0000","last_unstale":"2020-11-16T20:24:22.621039+0000","last_undegraded":"2020-11-16T20:24:22.621039+0000","last_fullsized":"2020-11-16T20:24:22.621039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:14.608430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:14.608430+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"38'20","reported_seq":"50","reported_epoch":"38","state":"active+clean+snaptrim","last_fresh":"2020-11-16T20:24:23.615993+0000","last_change":"2020-11-16T20:24:23.605426+0000","last_active":"2020-11-16T20:24:23.615993+0000","last_peered":"2020-11-16T20:24:23.615993+0000","last_clean":"2020-11-16T20:24:23.615993+0000","last_became_active":"2020-11-16T20:24:08.563031+0000","last_became_peered":"2020-11-16T20:24:08.563031+0000","last_unstale":"2020-11-16T20:24:23.615993+0000","last_undegraded":"2020-11-16T20:24:23.615993+0000","last_fullsized":"2020-11-16T20:24:23.615993+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":2424832,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":29,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624573+0000","last_change":"2020-11-16T20:23:44.501339+0000","last_active":"2020-11-16T20:24:22.624573+0000","last_peered":"2020-11-16T20:24:22.624573+0000","last_clean":"2020-11-16T20:24:22.624573+0000","last_became_active":"2020-11-16T20:23:44.500782+0000","last_became_peered":"2020-11-16T20:23:44.500782+0000","last_unstale":"2020-11-16T20:24:22.624573+0000","last_undegraded":"2020-11-16T20:24:22.624573+0000","last_fullsized":"2020-11-16T20:24:22.624573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"47","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.610872+0000","last_change":"2020-11-16T20:24:11.657961+0000","last_active":"2020-11-16T20:24:22.610872+0000","last_peered":"2020-11-16T20:24:22.610872+0000","last_clean":"2020-11-16T20:24:22.610872+0000","last_became_active":"2020-11-16T20:24:07.559654+0000","last_became_peered":"2020-11-16T20:24:07.559654+0000","last_unstale":"2020-11-16T20:24:22.610872+0000","last_undegraded":"2020-11-16T20:24:22.610872+0000","last_fullsized":"2020-11-16T20:24:22.610872+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:11.657774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:11.657774+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"38'30","reported_seq":"74","reported_epoch":"39","state":"active+clean+snaptrim","last_fresh":"2020-11-16T20:24:23.637172+0000","last_change":"2020-11-16T20:24:23.612594+0000","last_active":"2020-11-16T20:24:23.637172+0000","last_peered":"2020-11-16T20:24:23.637172+0000","last_clean":"2020-11-16T20:24:23.637172+0000","last_became_active":"2020-11-16T20:24:08.563332+0000","last_became_peered":"2020-11-16T20:24:08.563332+0000","last_unstale":"2020-11-16T20:24:23.637172+0000","last_undegraded":"2020-11-16T20:24:23.637172+0000","last_fullsized":"2020-11-16T20:24:23.637172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":2277376,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":2792,"num_write":15,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624607+0000","last_change":"2020-11-16T20:23:44.503055+0000","last_active":"2020-11-16T20:24:22.624607+0000","last_peered":"2020-11-16T20:24:22.624607+0000","last_clean":"2020-11-16T20:24:22.624607+0000","last_became_active":"2020-11-16T20:23:44.502588+0000","last_became_peered":"2020-11-16T20:23:44.502588+0000","last_unstale":"2020-11-16T20:24:22.624607+0000","last_undegraded":"2020-11-16T20:24:22.624607+0000","last_fullsized":"2020-11-16T20:24:22.624607+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":"54","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.612357+0000","last_change":"2020-11-16T20:24:15.472709+0000","last_active":"2020-11-16T20:24:22.612357+0000","last_peered":"2020-11-16T20:24:22.612357+0000","last_clean":"2020-11-16T20:24:22.612357+0000","last_became_active":"2020-11-16T20:24:14.526832+0000","last_became_peered":"2020-11-16T20:24:14.526832+0000","last_unstale":"2020-11-16T20:24:22.612357+0000","last_undegraded":"2020-11-16T20:24:22.612357+0000","last_fullsized":"2020-11-16T20:24:22.612357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:15.472519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:15.472519+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"39'13","reported_seq":"53","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.625129+0000","last_change":"2020-11-16T20:24:22.606898+0000","last_active":"2020-11-16T20:24:23.625129+0000","last_peered":"2020-11-16T20:24:23.625129+0000","last_clean":"2020-11-16T20:24:23.625129+0000","last_became_active":"2020-11-16T20:24:08.558421+0000","last_became_peered":"2020-11-16T20:24:08.558421+0000","last_unstale":"2020-11-16T20:24:23.625129+0000","last_undegraded":"2020-11-16T20:24:23.625129+0000","last_fullsized":"2020-11-16T20:24:23.625129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2244608,"num_objects":6,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":30,"num_write_kb":2192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.625384+0000","last_change":"2020-11-16T20:23:44.501820+0000","last_active":"2020-11-16T20:24:22.625384+0000","last_peered":"2020-11-16T20:24:22.625384+0000","last_clean":"2020-11-16T20:24:22.625384+0000","last_became_active":"2020-11-16T20:23:44.501424+0000","last_became_peered":"2020-11-16T20:23:44.501424+0000","last_unstale":"2020-11-16T20:24:22.625384+0000","last_undegraded":"2020-11-16T20:24:22.625384+0000","last_fullsized":"2020-11-16T20:24:22.625384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"14'2","reported_seq":"31","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.625012+0000","last_change":"2020-11-16T20:24:15.309347+0000","last_active":"2020-11-16T20:24:22.625012+0000","last_peered":"2020-11-16T20:24:22.625012+0000","last_clean":"2020-11-16T20:24:22.625012+0000","last_became_active":"2020-11-16T20:24:14.530884+0000","last_became_peered":"2020-11-16T20:24:14.530884+0000","last_unstale":"2020-11-16T20:24:22.625012+0000","last_undegraded":"2020-11-16T20:24:22.625012+0000","last_fullsized":"2020-11-16T20:24:22.625012+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:15.309247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:15.309247+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"38'23","reported_seq":"58","reported_epoch":"38","state":"active+clean+snaptrim","last_fresh":"2020-11-16T20:24:23.620532+0000","last_change":"2020-11-16T20:24:23.615618+0000","last_active":"2020-11-16T20:24:23.620532+0000","last_peered":"2020-11-16T20:24:23.620532+0000","last_clean":"2020-11-16T20:24:23.620532+0000","last_became_active":"2020-11-16T20:24:08.559749+0000","last_became_peered":"2020-11-16T20:24:08.559749+0000","last_unstale":"2020-11-16T20:24:23.620532+0000","last_undegraded":"2020-11-16T20:24:23.620532+0000","last_fullsized":"2020-11-16T20:24:23.620532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":1818624,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":5,"num_write":33,"num_write_kb":2784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":"29","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.614067+0000","last_change":"2020-11-16T20:23:44.598654+0000","last_active":"2020-11-16T20:24:22.614067+0000","last_peered":"2020-11-16T20:24:22.614067+0000","last_clean":"2020-11-16T20:24:22.614067+0000","last_became_active":"2020-11-16T20:23:44.597867+0000","last_became_peered":"2020-11-16T20:23:44.597867+0000","last_unstale":"2020-11-16T20:24:22.614067+0000","last_undegraded":"2020-11-16T20:24:22.614067+0000","last_fullsized":"2020-11-16T20:24:22.614067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"14'2","reported_seq":"31","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.614354+0000","last_change":"2020-11-16T20:24:19.329617+0000","last_active":"2020-11-16T20:24:22.614354+0000","last_peered":"2020-11-16T20:24:22.614354+0000","last_clean":"2020-11-16T20:24:22.614354+0000","last_became_active":"2020-11-16T20:24:14.529112+0000","last_became_peered":"2020-11-16T20:24:14.529112+0000","last_unstale":"2020-11-16T20:24:22.614354+0000","last_undegraded":"2020-11-16T20:24:22.614354+0000","last_fullsized":"2020-11-16T20:24:22.614354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:19.329415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:19.329415+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"39'10","reported_seq":"45","reported_epoch":"39","state":"active+clean","last_fresh":"2020-11-16T20:24:23.621985+0000","last_change":"2020-11-16T20:24:23.615500+0000","last_active":"2020-11-16T20:24:23.621985+0000","last_peered":"2020-11-16T20:24:23.621985+0000","last_clean":"2020-11-16T20:24:23.621985+0000","last_became_active":"2020-11-16T20:24:08.553683+0000","last_became_peered":"2020-11-16T20:24:08.553683+0000","last_unstale":"2020-11-16T20:24:23.621985+0000","last_undegraded":"2020-11-16T20:24:23.621985+0000","last_fullsized":"2020-11-16T20:24:23.621985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:24:07.520086+0000","last_clean_scrub_stamp":"2020-11-16T20:24:07.520086+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":950272,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":30,"num_write_kb":928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":"30","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.624443+0000","last_change":"2020-11-16T20:23:44.605343+0000","last_active":"2020-11-16T20:24:22.624443+0000","last_peered":"2020-11-16T20:24:22.624443+0000","last_clean":"2020-11-16T20:24:22.624443+0000","last_became_active":"2020-11-16T20:23:44.604919+0000","last_became_peered":"2020-11-16T20:23:44.604919+0000","last_unstale":"2020-11-16T20:24:22.624443+0000","last_undegraded":"2020-11-16T20:24:22.624443+0000","last_fullsized":"2020-11-16T20:24:22.624443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:43.466460+0000","last_clean_scrub_stamp":"2020-11-16T20:23:43.466460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"14'2","reported_seq":"25","reported_epoch":"38","state":"active+clean","last_fresh":"2020-11-16T20:24:22.617196+0000","last_change":"2020-11-16T20:24:16.437595+0000","last_active":"2020-11-16T20:24:22.617196+0000","last_peered":"2020-11-16T20:24:22.617196+0000","last_clean":"2020-11-16T20:24:22.617196+0000","last_became_active":"2020-11-16T20:24:14.527077+0000","last_became_peered":"2020-11-16T20:24:14.527077+0000","last_unstale":"2020-11-16T20:24:22.617196+0000","last_undegraded":"2020-11-16T20:24:22.617196+0000","last_fullsized":"2020-11-16T20:24:22.617196+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2020-11-16T20:24:16.437407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-16T20:23:36.602638+0000","last_clean_scrub_stamp":"2020-11-16T20:24:16.437407+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":25796608,"num_objects":58,"num_object_clones":10,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":149,"num_read_kb":12881,"num_write":288,"num_write_kb":27584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38694912,"data_stored":45563904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":22,"acting":22,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9532,"kb_used_data":4212,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94362308,"statfs":{"total":96636764160,"available":96627003392,"internally_reserved":0,"allocated":4313088,"data_stored":4615654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.835}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4220000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1960000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0619999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.913}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2909999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10648,"kb_used_data":5264,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94361192,"statfs":{"total":96636764160,"available":96625860608,"internally_reserved":0,"allocated":5390336,"data_stored":5668326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4849999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9390000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.786}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5670000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.714}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7920,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94363920,"statfs":{"total":96636764160,"available":96628654080,"internally_reserved":0,"allocated":2531328,"data_stored":2420198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6509999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11232,"kb_used_data":4632,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":94360608,"statfs":{"total":96636764160,"available":96625262592,"internally_reserved":0,"allocated":4743168,"data_stored":5508582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1640000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3279999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6120000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2879999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6749999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14192,"kb_used_data":7528,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":94357648,"statfs":{"total":96636764160,"available":96622231552,"internally_reserved":0,"allocated":7708672,"data_stored":9117158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4750000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7650000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.577}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.609}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5489999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7724,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94364116,"statfs":{"total":96636764160,"available":96628854784,"internally_reserved":0,"allocated":2658304,"data_stored":2420198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2559999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3439999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.399}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14532,"kb_used_data":8764,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94357308,"statfs":{"total":96636764160,"available":96621883392,"internally_reserved":0,"allocated":8974336,"data_stored":9735654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.796}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4390000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7210000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3239999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6549999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9548,"kb_used_data":3140,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94362292,"statfs":{"total":96636764160,"available":96626987008,"internally_reserved":0,"allocated":3215360,"data_stored":3751398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1909999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1389999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.669}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6230000000000002}]}]}]}} 2020-11-16T20:24:29.400 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-11-16T20:24:29.400 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-11-16T20:24:29.401 INFO:tasks.thrashosds.thrasher:inject_pause on 7 2020-11-16T20:24:29.401 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-11-16T20:24:29.401 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-11-16T20:24:29.401 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2020-11-16T20:24:29.444 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:29.445+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:29.475 INFO:tasks.rados.rados.0.smithi092.stdout:153: finishing write tid 5 to smithi09216568-46 2020-11-16T20:24:29.475 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 46 v 18 (ObjNum 82 snap 8 seq_num 82) dirty exists 2020-11-16T20:24:29.476 INFO:tasks.rados.rados.0.smithi092.stdout:153: left oid 46 (ObjNum 82 snap 8 seq_num 82) 2020-11-16T20:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:14 3.9s0 3:910fffa8:::smithi09216568-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-11-16T20:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.977530222, 2020-11-16T20:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033104548999999997, 2020-11-16T20:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 14 2020-11-16T20:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8910999999999999e-05 2020-11-16T20:24:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532325+0000", 2020-11-16T20:24:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2323e-05 2020-11-16T20:24:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532347+0000", 2020-11-16T20:24:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7090999999999999e-05 2020-11-16T20:24:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532374+0000", 2020-11-16T20:24:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5399999999999998e-06 2020-11-16T20:24:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532382+0000", 2020-11-16T20:24:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077685499999999999 2020-11-16T20:24:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540150+0000", 2020-11-16T20:24:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119175 2020-11-16T20:24:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540269+0000", 2020-11-16T20:24:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011674891 2020-11-16T20:24:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551944+0000", 2020-11-16T20:24:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010398199999999999 2020-11-16T20:24:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.552048+0000", 2020-11-16T20:24:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077962600000000002 2020-11-16T20:24:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.559845+0000", 2020-11-16T20:24:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028470140000000001 2020-11-16T20:24:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562692+0000", 2020-11-16T20:24:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002587677 2020-11-16T20:24:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565279+0000", 2020-11-16T20:24:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1135000000000001e-05 2020-11-16T20:24:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565370+0000", 2020-11-16T20:24:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033104548999999997 2020-11-16T20:24:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:15 3.9s0 3:910fffa8:::smithi09216568-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e30)", 2020-11-16T20:24:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.977362625, 2020-11-16T20:24:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037521408999999999, 2020-11-16T20:24:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15 2020-11-16T20:24:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9020000000000001e-05 2020-11-16T20:24:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532493+0000", 2020-11-16T20:24:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0396e-05 2020-11-16T20:24:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532503+0000", 2020-11-16T20:24:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9660000000000002e-05 2020-11-16T20:24:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532523+0000", 2020-11-16T20:24:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8280000000000003e-06 2020-11-16T20:24:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532530+0000", 2020-11-16T20:24:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077981070000000003 2020-11-16T20:24:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540329+0000", 2020-11-16T20:24:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8363000000000007e-05 2020-11-16T20:24:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540427+0000", 2020-11-16T20:24:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019650447000000001 2020-11-16T20:24:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560077+0000", 2020-11-16T20:24:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2565999999999994e-05 2020-11-16T20:24:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560170+0000", 2020-11-16T20:24:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052459309999999997 2020-11-16T20:24:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565416+0000", 2020-11-16T20:24:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1088999999999995e-05 2020-11-16T20:24:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565507+0000", 2020-11-16T20:24:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044480020000000004 2020-11-16T20:24:33.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.569955+0000", 2020-11-16T20:24:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037521408999999999 2020-11-16T20:24:33.675 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.676 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:23 3.ds0 3:b099cead:::smithi09216568-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-11-16T20:24:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.976918306, 2020-11-16T20:24:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025302294999999999, 2020-11-16T20:24:33.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23 2020-11-16T20:24:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5377999999999997e-05 2020-11-16T20:24:33.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532933+0000", 2020-11-16T20:24:33.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9703000000000001e-05 2020-11-16T20:24:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532953+0000", 2020-11-16T20:24:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2229e-05 2020-11-16T20:24:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532975+0000", 2020-11-16T20:24:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041310599999999998 2020-11-16T20:24:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537106+0000", 2020-11-16T20:24:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055730839999999999 2020-11-16T20:24:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542679+0000", 2020-11-16T20:24:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013964700000000001 2020-11-16T20:24:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542819+0000", 2020-11-16T20:24:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050541550000000003 2020-11-16T20:24:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.547873+0000", 2020-11-16T20:24:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253999999999993e-05 2020-11-16T20:24:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.547934+0000", 2020-11-16T20:24:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000519886 2020-11-16T20:24:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548454+0000", 2020-11-16T20:24:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066483360000000003 2020-11-16T20:24:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.555103+0000", 2020-11-16T20:24:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026224410000000001 2020-11-16T20:24:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557725+0000", 2020-11-16T20:24:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045512200000000002 2020-11-16T20:24:33.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558180+0000", 2020-11-16T20:24:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025302294999999999 2020-11-16T20:24:33.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:24 3.ds0 3:b099cead:::smithi09216568-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e30)", 2020-11-16T20:24:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.972624368, 2020-11-16T20:24:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025939752999999999, 2020-11-16T20:24:33.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 24 2020-11-16T20:24:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8302e-05 2020-11-16T20:24:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537210+0000", 2020-11-16T20:24:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5029999999999997e-06 2020-11-16T20:24:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537219+0000", 2020-11-16T20:24:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5136e-05 2020-11-16T20:24:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537234+0000", 2020-11-16T20:24:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2160000000000003e-06 2020-11-16T20:24:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537240+0000", 2020-11-16T20:24:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056701290000000003 2020-11-16T20:24:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542910+0000", 2020-11-16T20:24:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011685599999999999 2020-11-16T20:24:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543027+0000", 2020-11-16T20:24:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056655689999999996 2020-11-16T20:24:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548692+0000", 2020-11-16T20:24:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9920000000000003e-05 2020-11-16T20:24:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548742+0000", 2020-11-16T20:24:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094867259999999991 2020-11-16T20:24:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558229+0000", 2020-11-16T20:24:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8435000000000003e-05 2020-11-16T20:24:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558318+0000", 2020-11-16T20:24:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047939610000000002 2020-11-16T20:24:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563111+0000", 2020-11-16T20:24:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025939752999999999 2020-11-16T20:24:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:32 3.9s0 3:94376c81:::smithi09216568-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.970233906000001, 2020-11-16T20:24:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026430479999999999, 2020-11-16T20:24:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32 2020-11-16T20:24:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5041999999999999e-05 2020-11-16T20:24:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539617+0000", 2020-11-16T20:24:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2020-11-16T20:24:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539624+0000", 2020-11-16T20:24:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2020-11-16T20:24:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539636+0000", 2020-11-16T20:24:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.075e-06 2020-11-16T20:24:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539643+0000", 2020-11-16T20:24:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039536490000000001 2020-11-16T20:24:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543597+0000", 2020-11-16T20:24:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112988 2020-11-16T20:24:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543710+0000", 2020-11-16T20:24:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016926085 2020-11-16T20:24:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560636+0000", 2020-11-16T20:24:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8800000000000004e-05 2020-11-16T20:24:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560725+0000", 2020-11-16T20:24:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00080697699999999995 2020-11-16T20:24:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.561532+0000", 2020-11-16T20:24:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042868910000000001 2020-11-16T20:24:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565818+0000", 2020-11-16T20:24:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5162999999999998e-05 2020-11-16T20:24:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565904+0000", 2020-11-16T20:24:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9067999999999995e-05 2020-11-16T20:24:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565993+0000", 2020-11-16T20:24:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026430479999999999 2020-11-16T20:24:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:33 3.9s0 3:94376c81:::smithi09216568-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.970112112000001, 2020-11-16T20:24:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030497335, 2020-11-16T20:24:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33 2020-11-16T20:24:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8398000000000001e-05 2020-11-16T20:24:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539722+0000", 2020-11-16T20:24:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.993e-06 2020-11-16T20:24:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539728+0000", 2020-11-16T20:24:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2140000000000001e-05 2020-11-16T20:24:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539741+0000", 2020-11-16T20:24:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9979999999999997e-06 2020-11-16T20:24:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539745+0000", 2020-11-16T20:24:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004070204 2020-11-16T20:24:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543815+0000", 2020-11-16T20:24:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535999999999999 2020-11-16T20:24:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543920+0000", 2020-11-16T20:24:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018410834000000001 2020-11-16T20:24:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562331+0000", 2020-11-16T20:24:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4544999999999997e-05 2020-11-16T20:24:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562425+0000", 2020-11-16T20:24:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035976329999999998 2020-11-16T20:24:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.566023+0000", 2020-11-16T20:24:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2955999999999996e-05 2020-11-16T20:24:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.566116+0000", 2020-11-16T20:24:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040652739999999998 2020-11-16T20:24:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.570181+0000", 2020-11-16T20:24:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030497335 2020-11-16T20:24:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:35 3.9s0 3:9cc22e43:::smithi09216568-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.968939564999999, 2020-11-16T20:24:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11108446600000001, 2020-11-16T20:24:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 35 2020-11-16T20:24:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4415e-05 2020-11-16T20:24:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540911+0000", 2020-11-16T20:24:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.533e-06 2020-11-16T20:24:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540917+0000", 2020-11-16T20:24:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-05 2020-11-16T20:24:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540930+0000", 2020-11-16T20:24:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2769999999999999e-06 2020-11-16T20:24:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540934+0000", 2020-11-16T20:24:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010586527 2020-11-16T20:24:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551521+0000", 2020-11-16T20:24:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012027399999999999 2020-11-16T20:24:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551641+0000", 2020-11-16T20:24:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011083160999999999 2020-11-16T20:24:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562724+0000", 2020-11-16T20:24:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4109000000000004e-05 2020-11-16T20:24:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562808+0000", 2020-11-16T20:24:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000459751 2020-11-16T20:24:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563268+0000", 2020-11-16T20:24:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060973590000000001 2020-11-16T20:24:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.569365+0000", 2020-11-16T20:24:33.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082458486999999997 2020-11-16T20:24:33.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651824+0000", 2020-11-16T20:24:33.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117173 2020-11-16T20:24:33.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651941+0000", 2020-11-16T20:24:33.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11108446600000001 2020-11-16T20:24:33.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.764 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.765 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:36 3.9s0 3:9cc22e43:::smithi09216568-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:33.765 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:33.766 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.968820492000001, 2020-11-16T20:24:33.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.116868257, 2020-11-16T20:24:33.766 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:33.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.767 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.767 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36 2020-11-16T20:24:33.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:33.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:33.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9727000000000001e-05 2020-11-16T20:24:33.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541015+0000", 2020-11-16T20:24:33.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1739999999999997e-06 2020-11-16T20:24:33.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541022+0000", 2020-11-16T20:24:33.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1854e-05 2020-11-16T20:24:33.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541033+0000", 2020-11-16T20:24:33.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.799e-06 2020-11-16T20:24:33.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541037+0000", 2020-11-16T20:24:33.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019161628999999999 2020-11-16T20:24:33.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560199+0000", 2020-11-16T20:24:33.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102256 2020-11-16T20:24:33.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560301+0000", 2020-11-16T20:24:33.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032318479999999998 2020-11-16T20:24:33.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563533+0000", 2020-11-16T20:24:33.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2615999999999996e-05 2020-11-16T20:24:33.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563626+0000", 2020-11-16T20:24:33.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088372749 2020-11-16T20:24:33.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651998+0000", 2020-11-16T20:24:33.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113002 2020-11-16T20:24:33.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652111+0000", 2020-11-16T20:24:33.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057326030000000002 2020-11-16T20:24:33.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.657844+0000", 2020-11-16T20:24:33.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.116868257 2020-11-16T20:24:33.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.781 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.782 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=10, last_complete=31'6, committed=1, applied=1))", 2020-11-16T20:24:33.782 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:33.782 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.944178716, 2020-11-16T20:24:33.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086340027, 2020-11-16T20:24:33.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:33.783 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:33.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:33.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6184999999999999e-05 2020-11-16T20:24:33.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565664+0000", 2020-11-16T20:24:33.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7180000000000002e-06 2020-11-16T20:24:33.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565672+0000", 2020-11-16T20:24:33.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085916762999999993 2020-11-16T20:24:33.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651589+0000", 2020-11-16T20:24:33.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5119999999999997e-06 2020-11-16T20:24:33.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651596+0000", 2020-11-16T20:24:33.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3547000000000001e-05 2020-11-16T20:24:33.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651649+0000", 2020-11-16T20:24:33.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030830199999999999 2020-11-16T20:24:33.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651957+0000", 2020-11-16T20:24:33.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086340027 2020-11-16T20:24:33.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.792 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.792 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:85 3.9s0 3:9bc9395c:::smithi09216568-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:33.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:33.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.906079925, 2020-11-16T20:24:33.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048890797999999999, 2020-11-16T20:24:33.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:33.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 85 2020-11-16T20:24:33.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.794 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:33.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:33.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033801700000000003 2020-11-16T20:24:33.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604054+0000", 2020-11-16T20:24:33.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1904e-05 2020-11-16T20:24:33.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604066+0000", 2020-11-16T20:24:33.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2374e-05 2020-11-16T20:24:33.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604088+0000", 2020-11-16T20:24:33.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0937e-05 2020-11-16T20:24:33.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604099+0000", 2020-11-16T20:24:33.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6842999999999996e-05 2020-11-16T20:24:33.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604176+0000", 2020-11-16T20:24:33.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180261 2020-11-16T20:24:33.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604357+0000", 2020-11-16T20:24:33.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012281449999999999 2020-11-16T20:24:33.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605585+0000", 2020-11-16T20:24:33.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074810600000000003 2020-11-16T20:24:33.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613066+0000", 2020-11-16T20:24:33.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039419787999999997 2020-11-16T20:24:33.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652486+0000", 2020-11-16T20:24:33.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121469 2020-11-16T20:24:33.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652607+0000", 2020-11-16T20:24:33.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048890797999999999 2020-11-16T20:24:33.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.808 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:86 3.9s0 3:9bc9395c:::smithi09216568-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:33.810 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:33.810 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.905638245, 2020-11-16T20:24:33.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071173710000000001, 2020-11-16T20:24:33.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:33.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.811 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 86 2020-11-16T20:24:33.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:33.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:33.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7161999999999999e-05 2020-11-16T20:24:33.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604215+0000", 2020-11-16T20:24:33.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3989999999999998e-06 2020-11-16T20:24:33.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604222+0000", 2020-11-16T20:24:33.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3733999999999999e-05 2020-11-16T20:24:33.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604236+0000", 2020-11-16T20:24:33.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3090000000000003e-06 2020-11-16T20:24:33.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604241+0000", 2020-11-16T20:24:33.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074329820000000003 2020-11-16T20:24:33.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611674+0000", 2020-11-16T20:24:33.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117886 2020-11-16T20:24:33.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611792+0000", 2020-11-16T20:24:33.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040893269000000003 2020-11-16T20:24:33.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652686+0000", 2020-11-16T20:24:33.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9841999999999998e-05 2020-11-16T20:24:33.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652775+0000", 2020-11-16T20:24:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052300899999999995 2020-11-16T20:24:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653298+0000", 2020-11-16T20:24:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089837800000000002 2020-11-16T20:24:33.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654197+0000", 2020-11-16T20:24:33.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021031319999999999 2020-11-16T20:24:33.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675228+0000", 2020-11-16T20:24:33.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010342 2020-11-16T20:24:33.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675332+0000", 2020-11-16T20:24:33.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071173710000000001 2020-11-16T20:24:33.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:87 3.9s0 3:9bc9395c:::smithi09216568-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:33.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.905507417000001, 2020-11-16T20:24:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072859264000000007, 2020-11-16T20:24:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 87 2020-11-16T20:24:33.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.825 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:33.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:33.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1205000000000003e-05 2020-11-16T20:24:33.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604330+0000", 2020-11-16T20:24:33.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7689999999999999e-06 2020-11-16T20:24:33.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604337+0000", 2020-11-16T20:24:33.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3181e-05 2020-11-16T20:24:33.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604350+0000", 2020-11-16T20:24:33.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1000000000000003e-06 2020-11-16T20:24:33.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604355+0000", 2020-11-16T20:24:33.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074779859999999998 2020-11-16T20:24:33.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611833+0000", 2020-11-16T20:24:33.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087081400000000003 2020-11-16T20:24:33.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612704+0000", 2020-11-16T20:24:33.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040880316 2020-11-16T20:24:33.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653584+0000", 2020-11-16T20:24:33.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5697000000000003e-05 2020-11-16T20:24:33.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653670+0000", 2020-11-16T20:24:33.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021708466999999999 2020-11-16T20:24:33.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675378+0000", 2020-11-16T20:24:33.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7913999999999998e-05 2020-11-16T20:24:33.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675476+0000", 2020-11-16T20:24:33.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016718150000000001 2020-11-16T20:24:33.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.677148+0000", 2020-11-16T20:24:33.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072859264000000007 2020-11-16T20:24:33.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:103 3.9s0 3:98ee58dd:::smithi09216568-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:33.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:33.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.883868981999999, 2020-11-16T20:24:33.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037777976999999997, 2020-11-16T20:24:33.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:33.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 103 2020-11-16T20:24:33.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:33.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:33.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000376745 2020-11-16T20:24:33.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626304+0000", 2020-11-16T20:24:33.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0266e-05 2020-11-16T20:24:33.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626314+0000", 2020-11-16T20:24:33.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327e-05 2020-11-16T20:24:33.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626334+0000", 2020-11-16T20:24:33.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0482999999999999e-05 2020-11-16T20:24:33.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626345+0000", 2020-11-16T20:24:33.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7775999999999996e-05 2020-11-16T20:24:33.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626413+0000", 2020-11-16T20:24:33.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020824099999999999 2020-11-16T20:24:33.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626621+0000", 2020-11-16T20:24:33.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097088399999999998 2020-11-16T20:24:33.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.627592+0000", 2020-11-16T20:24:33.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074737739999999999 2020-11-16T20:24:33.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635066+0000", 2020-11-16T20:24:33.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028500588 2020-11-16T20:24:33.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663566+0000", 2020-11-16T20:24:33.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013889300000000001 2020-11-16T20:24:33.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663705+0000", 2020-11-16T20:24:33.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037777976999999997 2020-11-16T20:24:33.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.845 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:104 3.9s0 3:98ee58dd:::smithi09216568-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:33.845 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:33.845 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.883395291999999, 2020-11-16T20:24:33.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109118772, 2020-11-16T20:24:33.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:33.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104 2020-11-16T20:24:33.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.846 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:33.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:33.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4968999999999997e-05 2020-11-16T20:24:33.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626436+0000", 2020-11-16T20:24:33.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.33e-06 2020-11-16T20:24:33.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626440+0000", 2020-11-16T20:24:33.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1780000000000003e-06 2020-11-16T20:24:33.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626448+0000", 2020-11-16T20:24:33.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5559999999999998e-06 2020-11-16T20:24:33.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626454+0000", 2020-11-16T20:24:33.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082374119999999995 2020-11-16T20:24:33.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634691+0000", 2020-11-16T20:24:33.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6558999999999995e-05 2020-11-16T20:24:33.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634788+0000", 2020-11-16T20:24:33.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029028163999999999 2020-11-16T20:24:33.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663816+0000", 2020-11-16T20:24:33.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9767999999999998e-05 2020-11-16T20:24:33.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663906+0000", 2020-11-16T20:24:33.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088046700000000001 2020-11-16T20:24:33.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.664786+0000", 2020-11-16T20:24:33.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001328255 2020-11-16T20:24:33.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.666114+0000", 2020-11-16T20:24:33.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069334706999999995 2020-11-16T20:24:33.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735449+0000", 2020-11-16T20:24:33.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0407000000000005e-05 2020-11-16T20:24:33.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735520+0000", 2020-11-16T20:24:33.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109118772 2020-11-16T20:24:33.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.857 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:105 3.9s0 3:98ee58dd:::smithi09216568-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:33.857 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:33.857 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.883296101999999, 2020-11-16T20:24:33.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11144889600000001, 2020-11-16T20:24:33.857 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.857 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:33.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2020-11-16T20:24:33.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.858 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:33.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:33.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0299000000000002e-05 2020-11-16T20:24:33.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626540+0000", 2020-11-16T20:24:33.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3050000000000004e-06 2020-11-16T20:24:33.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626548+0000", 2020-11-16T20:24:33.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.878e-05 2020-11-16T20:24:33.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626566+0000", 2020-11-16T20:24:33.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3839999999999996e-06 2020-11-16T20:24:33.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626572+0000", 2020-11-16T20:24:33.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082488189999999992 2020-11-16T20:24:33.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634821+0000", 2020-11-16T20:24:33.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4552e-05 2020-11-16T20:24:33.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634905+0000", 2020-11-16T20:24:33.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030415658000000002 2020-11-16T20:24:33.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.665321+0000", 2020-11-16T20:24:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0865000000000005e-05 2020-11-16T20:24:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.665412+0000", 2020-11-16T20:24:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.070142494 2020-11-16T20:24:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735554+0000", 2020-11-16T20:24:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9005000000000007e-05 2020-11-16T20:24:33.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735633+0000", 2020-11-16T20:24:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023157350000000002 2020-11-16T20:24:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.737949+0000", 2020-11-16T20:24:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11144889600000001 2020-11-16T20:24:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:108 3.as0 3:5d07813d:::smithi09216568-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.878867865, 2020-11-16T20:24:33.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024149034999999999, 2020-11-16T20:24:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 108 2020-11-16T20:24:33.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:33.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4607999999999997e-05 2020-11-16T20:24:33.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630973+0000", 2020-11-16T20:24:33.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4680000000000003e-06 2020-11-16T20:24:33.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630980+0000", 2020-11-16T20:24:33.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019917399999999999 2020-11-16T20:24:33.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631179+0000", 2020-11-16T20:24:33.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4740000000000002e-06 2020-11-16T20:24:33.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631186+0000", 2020-11-16T20:24:33.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010229520000000001 2020-11-16T20:24:33.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641415+0000", 2020-11-16T20:24:33.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3351999999999997e-05 2020-11-16T20:24:33.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641499+0000", 2020-11-16T20:24:33.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074322040000000004 2020-11-16T20:24:33.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.648931+0000", 2020-11-16T20:24:33.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.166e-05 2020-11-16T20:24:33.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.649023+0000", 2020-11-16T20:24:33.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039121529999999998 2020-11-16T20:24:33.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652935+0000", 2020-11-16T20:24:33.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140617 2020-11-16T20:24:33.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653075+0000", 2020-11-16T20:24:33.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020018050000000002 2020-11-16T20:24:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.655077+0000", 2020-11-16T20:24:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024149034999999999 2020-11-16T20:24:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.880 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.880 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=24, last_complete=31'10, committed=1, applied=1))", 2020-11-16T20:24:33.880 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:33.880 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.834811125, 2020-11-16T20:24:33.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060547953000000002, 2020-11-16T20:24:33.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:33.881 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:33.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:33.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6891000000000003e-05 2020-11-16T20:24:33.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675022+0000", 2020-11-16T20:24:33.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3489999999999998e-06 2020-11-16T20:24:33.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675031+0000", 2020-11-16T20:24:33.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060268058999999999 2020-11-16T20:24:33.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735299+0000", 2020-11-16T20:24:33.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7539999999999995e-06 2020-11-16T20:24:33.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735308+0000", 2020-11-16T20:24:33.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7015999999999998e-05 2020-11-16T20:24:33.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735355+0000", 2020-11-16T20:24:33.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017788400000000001 2020-11-16T20:24:33.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735533+0000", 2020-11-16T20:24:33.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060547953000000002 2020-11-16T20:24:33.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.886 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.887 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:153 3.9s0 3:94376c81:::smithi09216568-11:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:33.887 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:33.887 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.799006371000001, 2020-11-16T20:24:33.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056909992999999999, 2020-11-16T20:24:33.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.887 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:33.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.888 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 153 2020-11-16T20:24:33.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.888 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:33.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:33.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6903999999999998e-05 2020-11-16T20:24:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710827+0000", 2020-11-16T20:24:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0011e-05 2020-11-16T20:24:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710837+0000", 2020-11-16T20:24:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4769e-05 2020-11-16T20:24:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710851+0000", 2020-11-16T20:24:33.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1129999999999997e-06 2020-11-16T20:24:33.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710858+0000", 2020-11-16T20:24:33.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6598999999999998e-05 2020-11-16T20:24:33.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710914+0000", 2020-11-16T20:24:33.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110771 2020-11-16T20:24:33.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711025+0000", 2020-11-16T20:24:33.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011061122 2020-11-16T20:24:33.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:33.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.722086+0000", 2020-11-16T20:24:33.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055101880000000001 2020-11-16T20:24:33.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:33.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.727596+0000", 2020-11-16T20:24:33.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039941002000000003 2020-11-16T20:24:33.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:33.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767537+0000", 2020-11-16T20:24:33.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000162514 2020-11-16T20:24:33.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767700+0000", 2020-11-16T20:24:33.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056909992999999999 2020-11-16T20:24:33.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:154 3.9s0 3:94376c81:::smithi09216568-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:33.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:33.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.798886907, 2020-11-16T20:24:33.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058295893000000001, 2020-11-16T20:24:33.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:33.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:33.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:33.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:33.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2020-11-16T20:24:33.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:33.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:33.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7756999999999998e-05 2020-11-16T20:24:33.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710947+0000", 2020-11-16T20:24:33.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7429999999999996e-06 2020-11-16T20:24:33.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710954+0000", 2020-11-16T20:24:33.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2998e-05 2020-11-16T20:24:33.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710967+0000", 2020-11-16T20:24:33.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8250000000000004e-06 2020-11-16T20:24:33.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710972+0000", 2020-11-16T20:24:33.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022201499999999999 2020-11-16T20:24:33.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711194+0000", 2020-11-16T20:24:33.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1608e-05 2020-11-16T20:24:33.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2020-11-16T20:24:33.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711225+0000", 2020-11-16T20:24:33.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012795857000000001 2020-11-16T20:24:33.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724021+0000", 2020-11-16T20:24:33.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7220999999999998e-05 2020-11-16T20:24:33.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:33.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724098+0000", 2020-11-16T20:24:33.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044522785000000002 2020-11-16T20:24:33.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768621+0000", 2020-11-16T20:24:33.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026034400000000003 2020-11-16T20:24:33.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:33.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768881+0000", 2020-11-16T20:24:33.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032373999999999999 2020-11-16T20:24:33.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.769205+0000", 2020-11-16T20:24:33.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058295893000000001 2020-11-16T20:24:33.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.908 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=26, last_complete=31'11, committed=1, applied=1))", 2020-11-16T20:24:33.908 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:33.908 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.785447206000001, 2020-11-16T20:24:33.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044143454999999998, 2020-11-16T20:24:33.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:33.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:33.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:33.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:33.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:33.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:33.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:33.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:33.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7308999999999999e-05 2020-11-16T20:24:33.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:33.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724376+0000", 2020-11-16T20:24:33.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9070000000000004e-06 2020-11-16T20:24:33.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:33.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724382+0000", 2020-11-16T20:24:33.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042942860999999999 2020-11-16T20:24:33.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767325+0000", 2020-11-16T20:24:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2480000000000001e-05 2020-11-16T20:24:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767337+0000", 2020-11-16T20:24:33.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.177e-05 2020-11-16T20:24:33.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:33.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767399+0000", 2020-11-16T20:24:33.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010931280000000001 2020-11-16T20:24:33.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:33.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:33.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:33.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768492+0000", 2020-11-16T20:24:33.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044143454999999998 2020-11-16T20:24:33.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:33.915 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:33.915 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:33.915 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2020-11-16T20:24:33.916 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-11-16T20:24:33.921 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:33.922 INFO:teuthology.orchestra.run.smithi013.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-11-16T20:24:33.922 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:33.922 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:33.912+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:33.922 INFO:tasks.rados.rados.0.smithi092.stdout:151: done (2 left) 2020-11-16T20:24:33.922 INFO:tasks.rados.rados.0.smithi092.stdout:153: done (1 left) 2020-11-16T20:24:33.923 INFO:tasks.rados.rados.0.smithi092.stdout:154: done (0 left) 2020-11-16T20:24:33.923 INFO:tasks.rados.rados.0.smithi092.stdout:155: read oid 24 snap -1 2020-11-16T20:24:33.923 INFO:tasks.rados.rados.0.smithi092.stdout:155: expect (ObjNum 66 snap 5 seq_num 66) 2020-11-16T20:24:33.923 INFO:tasks.rados.rados.0.smithi092.stdout:156: delete oid 38 current snap is 9 2020-11-16T20:24:33.923 INFO:tasks.rados.rados.0.smithi092.stdout:155: expect (ObjNum 66 snap 5 seq_num 66) 2020-11-16T20:24:33.923 INFO:tasks.rados.rados.0.smithi092.stdout:155: done (1 left) 2020-11-16T20:24:33.923 INFO:tasks.rados.rados.0.smithi092.stdout:156: done (0 left) 2020-11-16T20:24:33.924 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 15 current snap is 9 2020-11-16T20:24:33.924 INFO:tasks.rados.rados.0.smithi092.stdout:157: seq_num 84 ranges {0=614400,614400=434176,1048576=368640} 2020-11-16T20:24:33.924 INFO:tasks.rados.rados.0.smithi092.stdout:157: writing smithi09216568-15 from 0 to 614400 tid 1 2020-11-16T20:24:33.924 INFO:tasks.rados.rados.0.smithi092.stdout:157: writing smithi09216568-15 from 614400 to 1048576 tid 2 2020-11-16T20:24:33.924 INFO:tasks.rados.rados.0.smithi092.stdout:157: writing smithi09216568-15 from 1048576 to 1417216 tid 3 2020-11-16T20:24:33.924 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 39 current snap is 9 2020-11-16T20:24:33.924 INFO:tasks.rados.rados.0.smithi092.stdout:158: seq_num 85 ranges {1351680=630784,1982464=720896,2703360=81920} 2020-11-16T20:24:33.925 INFO:tasks.rados.rados.0.smithi092.stdout:158: writing smithi09216568-39 from 1351680 to 1982464 tid 1 2020-11-16T20:24:33.925 INFO:tasks.rados.rados.0.smithi092.stdout:158: writing smithi09216568-39 from 1982464 to 2703360 tid 2 2020-11-16T20:24:33.925 INFO:tasks.rados.rados.0.smithi092.stdout:158: writing smithi09216568-39 from 2703360 to 2785280 tid 3 2020-11-16T20:24:33.925 INFO:tasks.rados.rados.0.smithi092.stdout:159: delete oid 43 current snap is 9 2020-11-16T20:24:33.925 INFO:tasks.rados.rados.0.smithi092.stdout:159: done (2 left) 2020-11-16T20:24:33.925 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 29 current snap is 9 2020-11-16T20:24:33.926 INFO:tasks.rados.rados.0.smithi092.stdout:160: seq_num 87 ranges {475136=663552,1138688=491520,1630208=311296} 2020-11-16T20:24:33.926 INFO:tasks.rados.rados.0.smithi092.stdout:160: writing smithi09216568-29 from 475136 to 1138688 tid 1 2020-11-16T20:24:33.926 INFO:tasks.rados.rados.0.smithi092.stdout:160: writing smithi09216568-29 from 1138688 to 1630208 tid 2 2020-11-16T20:24:33.926 INFO:tasks.rados.rados.0.smithi092.stdout:160: writing smithi09216568-29 from 1630208 to 1941504 tid 3 2020-11-16T20:24:33.926 INFO:tasks.rados.rados.0.smithi092.stdout:161: snap_create 2020-11-16T20:24:33.926 INFO:tasks.rados.rados.0.smithi092.stdout:157: finishing write tid 1 to smithi09216568-15 2020-11-16T20:24:33.926 INFO:tasks.rados.rados.0.smithi092.stdout:157: finishing write tid 2 to smithi09216568-15 2020-11-16T20:24:33.927 INFO:tasks.rados.rados.0.smithi092.stdout:157: finishing write tid 3 to smithi09216568-15 2020-11-16T20:24:33.927 INFO:tasks.rados.rados.0.smithi092.stdout:157: finishing write tid 4 to smithi09216568-15 2020-11-16T20:24:33.927 INFO:tasks.rados.rados.0.smithi092.stdout:157: finishing write tid 5 to smithi09216568-15 2020-11-16T20:24:33.927 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 15 v 37 (ObjNum 84 snap 9 seq_num 84) dirty exists 2020-11-16T20:24:33.927 INFO:tasks.rados.rados.0.smithi092.stdout:157: left oid 15 (ObjNum 84 snap 9 seq_num 84) 2020-11-16T20:24:33.927 INFO:tasks.rados.rados.0.smithi092.stdout:158: finishing write tid 1 to smithi09216568-39 2020-11-16T20:24:33.927 INFO:tasks.rados.rados.0.smithi092.stdout:158: finishing write tid 2 to smithi09216568-39 2020-11-16T20:24:33.927 INFO:tasks.rados.rados.0.smithi092.stdout:158: finishing write tid 3 to smithi09216568-39 2020-11-16T20:24:33.928 INFO:tasks.rados.rados.0.smithi092.stdout:158: finishing write tid 4 to smithi09216568-39 2020-11-16T20:24:33.928 INFO:tasks.rados.rados.0.smithi092.stdout:158: finishing write tid 5 to smithi09216568-39 2020-11-16T20:24:33.928 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 39 v 36 (ObjNum 85 snap 9 seq_num 85) dirty exists 2020-11-16T20:24:33.928 INFO:tasks.rados.rados.0.smithi092.stdout:158: left oid 39 (ObjNum 85 snap 9 seq_num 85) 2020-11-16T20:24:33.928 INFO:tasks.rados.rados.0.smithi092.stdout:160: finishing write tid 1 to smithi09216568-29 2020-11-16T20:24:33.928 INFO:tasks.rados.rados.0.smithi092.stdout:160: finishing write tid 2 to smithi09216568-29 2020-11-16T20:24:33.928 INFO:tasks.rados.rados.0.smithi092.stdout:160: finishing write tid 3 to smithi09216568-29 2020-11-16T20:24:33.928 INFO:tasks.rados.rados.0.smithi092.stdout:160: finishing write tid 4 to smithi09216568-29 2020-11-16T20:24:33.929 INFO:tasks.rados.rados.0.smithi092.stdout:160: finishing write tid 5 to smithi09216568-29 2020-11-16T20:24:33.929 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 29 v 52 (ObjNum 87 snap 9 seq_num 87) dirty exists 2020-11-16T20:24:33.929 INFO:tasks.rados.rados.0.smithi092.stdout:160: left oid 29 (ObjNum 87 snap 9 seq_num 87) 2020-11-16T20:24:33.929 INFO:tasks.rados.rados.0.smithi092.stdout:157: done (3 left) 2020-11-16T20:24:33.929 INFO:tasks.rados.rados.0.smithi092.stdout:158: done (2 left) 2020-11-16T20:24:33.929 INFO:tasks.rados.rados.0.smithi092.stdout:160: done (1 left) 2020-11-16T20:24:33.929 INFO:tasks.rados.rados.0.smithi092.stdout:161: done (0 left) 2020-11-16T20:24:33.930 INFO:tasks.rados.rados.0.smithi092.stdout:162: read oid 47 snap -1 2020-11-16T20:24:33.930 INFO:tasks.rados.rados.0.smithi092.stdout:162: expect (ObjNum 46 snap 0 seq_num 46) 2020-11-16T20:24:33.930 INFO:tasks.rados.rados.0.smithi092.stdout:163: read oid 35 snap -1 2020-11-16T20:24:33.930 INFO:tasks.rados.rados.0.smithi092.stdout:163: expect (ObjNum 34 snap 0 seq_num 34) 2020-11-16T20:24:33.930 INFO:tasks.rados.rados.0.smithi092.stdout:164: setattr oid 32 current snap is 10 2020-11-16T20:24:33.930 INFO:tasks.rados.rados.0.smithi092.stdout:165: read oid 11 snap -1 2020-11-16T20:24:33.930 INFO:tasks.rados.rados.0.smithi092.stdout:165: expect (ObjNum 59 snap 5 seq_num 59) 2020-11-16T20:24:33.930 INFO:tasks.rados.rados.0.smithi092.stdout:166: snap_remove snap 6 2020-11-16T20:24:33.931 INFO:tasks.rados.rados.0.smithi092.stdout:166: done (4 left) 2020-11-16T20:24:33.931 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 40 current snap is 10 2020-11-16T20:24:33.931 INFO:tasks.rados.rados.0.smithi092.stdout:167: seq_num 88 ranges {663552=679936,1343488=778240,2121728=24576} 2020-11-16T20:24:33.931 INFO:tasks.rados.rados.0.smithi092.stdout:167: writing smithi09216568-40 from 663552 to 1343488 tid 1 2020-11-16T20:24:33.931 INFO:tasks.rados.rados.0.smithi092.stdout:167: writing smithi09216568-40 from 1343488 to 2121728 tid 2 2020-11-16T20:24:33.931 INFO:tasks.rados.rados.0.smithi092.stdout:167: writing smithi09216568-40 from 2121728 to 2146304 tid 3 2020-11-16T20:24:33.931 INFO:tasks.rados.rados.0.smithi092.stdout:162: expect (ObjNum 46 snap 0 seq_num 46) 2020-11-16T20:24:33.931 INFO:tasks.rados.rados.0.smithi092.stdout:163: expect (ObjNum 34 snap 0 seq_num 34) 2020-11-16T20:24:33.932 INFO:tasks.rados.rados.0.smithi092.stdout:165: expect (ObjNum 59 snap 5 seq_num 59) 2020-11-16T20:24:33.932 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 32 v 16 (ObjNum 73 snap 6 seq_num 73) dirty exists 2020-11-16T20:24:33.932 INFO:tasks.rados.rados.0.smithi092.stdout:167: finishing write tid 1 to smithi09216568-40 2020-11-16T20:24:33.932 INFO:tasks.rados.rados.0.smithi092.stdout:167: finishing write tid 2 to smithi09216568-40 2020-11-16T20:24:33.932 INFO:tasks.rados.rados.0.smithi092.stdout:167: finishing write tid 3 to smithi09216568-40 2020-11-16T20:24:33.932 INFO:tasks.rados.rados.0.smithi092.stdout:167: finishing write tid 4 to smithi09216568-40 2020-11-16T20:24:33.932 INFO:tasks.rados.rados.0.smithi092.stdout:167: finishing write tid 5 to smithi09216568-40 2020-11-16T20:24:33.933 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 40 v 44 (ObjNum 88 snap 10 seq_num 88) dirty exists 2020-11-16T20:24:33.933 INFO:tasks.rados.rados.0.smithi092.stdout:167: left oid 40 (ObjNum 88 snap 10 seq_num 88) 2020-11-16T20:24:33.933 INFO:tasks.rados.rados.0.smithi092.stdout:162: done (4 left) 2020-11-16T20:24:33.933 INFO:tasks.rados.rados.0.smithi092.stdout:163: done (3 left) 2020-11-16T20:24:33.933 INFO:tasks.rados.rados.0.smithi092.stdout:164: done (2 left) 2020-11-16T20:24:33.933 INFO:tasks.rados.rados.0.smithi092.stdout:165: done (1 left) 2020-11-16T20:24:33.933 INFO:tasks.rados.rados.0.smithi092.stdout:167: done (0 left) 2020-11-16T20:24:33.933 INFO:tasks.rados.rados.0.smithi092.stdout:168: read oid 28 snap -1 2020-11-16T20:24:33.934 INFO:tasks.rados.rados.0.smithi092.stdout:168: expect (ObjNum 63 snap 5 seq_num 63) 2020-11-16T20:24:33.934 INFO:tasks.rados.rados.0.smithi092.stdout:169: rmattr oid 10 current snap is 10 2020-11-16T20:24:33.934 INFO:tasks.rados.rados.0.smithi092.stdout:169: done (1 left) 2020-11-16T20:24:33.934 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 41 current snap is 10 2020-11-16T20:24:33.934 INFO:tasks.rados.rados.0.smithi092.stdout:170: seq_num 89 ranges {0=696320,696320=655360,1351680=147456} 2020-11-16T20:24:33.934 INFO:tasks.rados.rados.0.smithi092.stdout:170: writing smithi09216568-41 from 0 to 696320 tid 1 2020-11-16T20:24:33.934 INFO:tasks.rados.rados.0.smithi092.stdout:170: writing smithi09216568-41 from 696320 to 1351680 tid 2 2020-11-16T20:24:33.935 INFO:tasks.rados.rados.0.smithi092.stdout:170: writing smithi09216568-41 from 1351680 to 1499136 tid 3 2020-11-16T20:24:33.935 INFO:tasks.rados.rados.0.smithi092.stdout:171: copy_from oid 21 from oid 3 current snap is 10 2020-11-16T20:24:33.935 INFO:tasks.rados.rados.0.smithi092.stdout:172: snap_create 2020-11-16T20:24:33.935 INFO:tasks.rados.rados.0.smithi092.stdout:168: expect (ObjNum 63 snap 5 seq_num 63) 2020-11-16T20:24:33.935 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 1 to smithi09216568-41 2020-11-16T20:24:33.935 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 2 to smithi09216568-41 2020-11-16T20:24:33.935 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 3 to smithi09216568-41 2020-11-16T20:24:33.935 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 4 to smithi09216568-41 2020-11-16T20:24:33.936 INFO:tasks.rados.rados.0.smithi092.stdout:170: finishing write tid 5 to smithi09216568-41 2020-11-16T20:24:33.936 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 41 v 48 (ObjNum 89 snap 10 seq_num 89) dirty exists 2020-11-16T20:24:33.936 INFO:tasks.rados.rados.0.smithi092.stdout:170: left oid 41 (ObjNum 89 snap 10 seq_num 89) 2020-11-16T20:24:33.936 INFO:tasks.rados.rados.0.smithi092.stdout:171: finishing copy_from to smithi09216568-21 2020-11-16T20:24:33.936 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 21 v 26 (ObjNum 79 snap 7 seq_num 79) dirty exists 2020-11-16T20:24:33.936 INFO:tasks.rados.rados.0.smithi092.stdout:171: finishing copy_from racing read to smithi09216568-21 2020-11-16T20:24:33.936 INFO:tasks.rados.rados.0.smithi092.stdout:168: done (3 left) 2020-11-16T20:24:33.936 INFO:tasks.rados.rados.0.smithi092.stdout:170: done (2 left) 2020-11-16T20:24:33.937 INFO:tasks.rados.rados.0.smithi092.stdout:171: done (1 left) 2020-11-16T20:24:33.937 INFO:tasks.rados.rados.0.smithi092.stdout:172: done (0 left) 2020-11-16T20:24:33.937 INFO:tasks.rados.rados.0.smithi092.stdout:173: copy_from oid 20 from oid 21 current snap is 11 2020-11-16T20:24:33.937 INFO:tasks.rados.rados.0.smithi092.stdout:174: snap_create 2020-11-16T20:24:33.937 INFO:tasks.rados.rados.0.smithi092.stdout:173: finishing copy_from to smithi09216568-20 2020-11-16T20:24:33.937 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 20 v 26 (ObjNum 79 snap 7 seq_num 79) dirty exists 2020-11-16T20:24:33.937 INFO:tasks.rados.rados.0.smithi092.stdout:173: finishing copy_from racing read to smithi09216568-20 2020-11-16T20:24:33.938 INFO:tasks.rados.rados.0.smithi092.stdout:173: done (1 left) 2020-11-16T20:24:33.938 INFO:tasks.rados.rados.0.smithi092.stdout:174: done (0 left) 2020-11-16T20:24:33.938 INFO:tasks.rados.rados.0.smithi092.stdout:175: read oid 22 snap -1 2020-11-16T20:24:33.938 INFO:tasks.rados.rados.0.smithi092.stdout:175: expect deleted 2020-11-16T20:24:33.938 INFO:tasks.rados.rados.0.smithi092.stdout:176: snap_remove snap 11 2020-11-16T20:24:33.939 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-11-16T20:24:34.017 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:34.017+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:34.087 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:34.087 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:34.087 INFO:teuthology.orchestra.run.smithi013.stdout: "num_ops": 0 2020-11-16T20:24:34.087 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:34.098 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-11-16T20:24:34.118 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:34.145 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:34.144+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:34.146 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:34.173 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:34.200 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:34.213 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:34.214 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:34.214 INFO:teuthology.orchestra.run.smithi013.stdout: "complaint_time": 30, 2020-11-16T20:24:34.214 INFO:teuthology.orchestra.run.smithi013.stdout: "num_blocked_ops": 0 2020-11-16T20:24:34.214 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:34.218 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:34.217+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:34.224 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-11-16T20:24:34.228 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:34.267 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:34.294 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:34.320 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:34.321+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:34.391 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:34.392 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:34.402 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-11-16T20:24:34.420 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:34.421+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:34.520 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:34.521 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:34.521 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:34.521+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:34.530 INFO:tasks.rados.rados.0.smithi092.stdout:176: done (1 left) 2020-11-16T20:24:34.530 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 43 current snap is 12 2020-11-16T20:24:34.530 INFO:tasks.rados.rados.0.smithi092.stdout:177: seq_num 90 ranges {0=712704,712704=540672,1253376=262144} 2020-11-16T20:24:34.533 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-11-16T20:24:34.534 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2020-11-16T20:24:34.536 INFO:tasks.rados.rados.0.smithi092.stdout:177: writing smithi09216568-43 from 0 to 712704 tid 1 2020-11-16T20:24:34.541 INFO:tasks.rados.rados.0.smithi092.stdout:177: writing smithi09216568-43 from 712704 to 1253376 tid 2 2020-11-16T20:24:34.543 INFO:tasks.rados.rados.0.smithi092.stdout:177: writing smithi09216568-43 from 1253376 to 1515520 tid 3 2020-11-16T20:24:34.544 INFO:tasks.rados.rados.0.smithi092.stdout:175: done (1 left) 2020-11-16T20:24:34.544 INFO:tasks.rados.rados.0.smithi092.stdout:178: read oid 20 snap -1 2020-11-16T20:24:34.544 INFO:tasks.rados.rados.0.smithi092.stdout:178: expect (ObjNum 79 snap 7 seq_num 79) 2020-11-16T20:24:34.545 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 37 current snap is 12 2020-11-16T20:24:34.545 INFO:tasks.rados.rados.0.smithi092.stdout:179: seq_num 91 ranges {958464=729088,1687552=425984,2113536=376832} 2020-11-16T20:24:34.557 INFO:tasks.rados.rados.0.smithi092.stdout:179: writing smithi09216568-37 from 958464 to 1687552 tid 1 2020-11-16T20:24:34.560 INFO:tasks.rados.rados.0.smithi092.stdout:179: writing smithi09216568-37 from 1687552 to 2113536 tid 2 2020-11-16T20:24:34.562 INFO:tasks.rados.rados.0.smithi092.stdout:179: writing smithi09216568-37 from 2113536 to 2490368 tid 3 2020-11-16T20:24:34.563 INFO:tasks.rados.rados.0.smithi092.stdout:180: read oid 16 snap -1 2020-11-16T20:24:34.563 INFO:tasks.rados.rados.0.smithi092.stdout:180: expect (ObjNum 75 snap 6 seq_num 75) 2020-11-16T20:24:34.563 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 11 current snap is 12 2020-11-16T20:24:34.563 INFO:tasks.rados.rados.0.smithi092.stdout:181: seq_num 92 ranges {1089536=745472,1835008=761856,2596864=40960} 2020-11-16T20:24:34.573 INFO:tasks.rados.rados.0.smithi092.stdout:181: writing smithi09216568-11 from 1089536 to 1835008 tid 1 2020-11-16T20:24:34.578 INFO:tasks.rados.rados.0.smithi092.stdout:181: writing smithi09216568-11 from 1835008 to 2596864 tid 2 2020-11-16T20:24:34.579 INFO:tasks.rados.rados.0.smithi092.stdout:181: writing smithi09216568-11 from 2596864 to 2637824 tid 3 2020-11-16T20:24:34.579 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 27 current snap is 12 2020-11-16T20:24:34.579 INFO:tasks.rados.rados.0.smithi092.stdout:182: seq_num 93 ranges {0=761856,761856=647168,1409024=155648} 2020-11-16T20:24:34.584 INFO:tasks.rados.rados.0.smithi092.stdout:182: writing smithi09216568-27 from 0 to 761856 tid 1 2020-11-16T20:24:34.588 INFO:tasks.rados.rados.0.smithi092.stdout:182: writing smithi09216568-27 from 761856 to 1409024 tid 2 2020-11-16T20:24:34.589 INFO:tasks.rados.rados.0.smithi092.stdout:182: writing smithi09216568-27 from 1409024 to 1564672 tid 3 2020-11-16T20:24:34.589 INFO:tasks.rados.rados.0.smithi092.stdout:183: delete oid 23 current snap is 12 2020-11-16T20:24:34.590 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 1 to smithi09216568-43 2020-11-16T20:24:34.590 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 2 to smithi09216568-43 2020-11-16T20:24:34.590 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 3 to smithi09216568-43 2020-11-16T20:24:34.590 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 4 to smithi09216568-43 2020-11-16T20:24:34.590 INFO:tasks.rados.rados.0.smithi092.stdout:177: finishing write tid 5 to smithi09216568-43 2020-11-16T20:24:34.590 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 43 v 36 (ObjNum 90 snap 12 seq_num 90) dirty exists 2020-11-16T20:24:34.591 INFO:tasks.rados.rados.0.smithi092.stdout:177: left oid 43 (ObjNum 90 snap 12 seq_num 90) 2020-11-16T20:24:34.602 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 1 to smithi09216568-37 2020-11-16T20:24:34.603 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 2 to smithi09216568-37 2020-11-16T20:24:34.603 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 3 to smithi09216568-37 2020-11-16T20:24:34.603 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 4 to smithi09216568-37 2020-11-16T20:24:34.609 INFO:tasks.rados.rados.0.smithi092.stdout:179: finishing write tid 5 to smithi09216568-37 2020-11-16T20:24:34.610 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 37 v 25 (ObjNum 91 snap 12 seq_num 91) dirty exists 2020-11-16T20:24:34.610 INFO:tasks.rados.rados.0.smithi092.stdout:179: left oid 37 (ObjNum 91 snap 12 seq_num 91) 2020-11-16T20:24:34.610 INFO:tasks.rados.rados.0.smithi092.stdout:180: expect (ObjNum 75 snap 6 seq_num 75) 2020-11-16T20:24:34.621 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:34.621+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:34.662 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:34.663 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:34.672 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-11-16T20:24:34.710 INFO:tasks.rados.rados.0.smithi092.stdout:182: finishing write tid 1 to smithi09216568-27 2020-11-16T20:24:34.711 INFO:tasks.rados.rados.0.smithi092.stdout:181: finishing write tid 1 to smithi09216568-11 2020-11-16T20:24:34.711 INFO:tasks.rados.rados.0.smithi092.stdout:181: finishing write tid 2 to smithi09216568-11 2020-11-16T20:24:34.711 INFO:tasks.rados.rados.0.smithi092.stdout:181: finishing write tid 3 to smithi09216568-11 2020-11-16T20:24:34.711 INFO:tasks.rados.rados.0.smithi092.stdout:181: finishing write tid 4 to smithi09216568-11 2020-11-16T20:24:34.711 INFO:tasks.rados.rados.0.smithi092.stdout:182: finishing write tid 2 to smithi09216568-27 2020-11-16T20:24:34.711 INFO:tasks.rados.rados.0.smithi092.stdout:181: finishing write tid 5 to smithi09216568-11 2020-11-16T20:24:34.711 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 11 v 61 (ObjNum 92 snap 12 seq_num 92) dirty exists 2020-11-16T20:24:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:181: left oid 11 (ObjNum 92 snap 12 seq_num 92) 2020-11-16T20:24:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:182: finishing write tid 3 to smithi09216568-27 2020-11-16T20:24:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:182: finishing write tid 4 to smithi09216568-27 2020-11-16T20:24:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:182: finishing write tid 5 to smithi09216568-27 2020-11-16T20:24:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 27 v 23 (ObjNum 93 snap 12 seq_num 93) dirty exists 2020-11-16T20:24:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:182: left oid 27 (ObjNum 93 snap 12 seq_num 93) 2020-11-16T20:24:34.712 INFO:tasks.rados.rados.0.smithi092.stdout:177: done (6 left) 2020-11-16T20:24:34.713 INFO:tasks.rados.rados.0.smithi092.stdout:179: done (5 left) 2020-11-16T20:24:34.713 INFO:tasks.rados.rados.0.smithi092.stdout:180: done (4 left) 2020-11-16T20:24:34.713 INFO:tasks.rados.rados.0.smithi092.stdout:181: done (3 left) 2020-11-16T20:24:34.713 INFO:tasks.rados.rados.0.smithi092.stdout:182: done (2 left) 2020-11-16T20:24:34.713 INFO:tasks.rados.rados.0.smithi092.stdout:183: done (1 left) 2020-11-16T20:24:34.714 INFO:tasks.rados.rados.0.smithi092.stdout:184: snap_create 2020-11-16T20:24:34.722 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:34.721+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:34.751 INFO:tasks.rados.rados.0.smithi092.stdout:178: expect (ObjNum 79 snap 7 seq_num 79) 2020-11-16T20:24:34.787 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:34.787 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:34.797 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-11-16T20:24:34.823 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:34.821+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:34.917 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:34.917 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:34.924 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:34.925+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:34.927 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-11-16T20:24:35.025 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:35.025+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:35.047 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:35.048 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:35.058 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-11-16T20:24:35.126 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:35.124+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:35.134 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-11-16T20:24:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-11-16T20:24:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-11-16T20:24:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-11-16T20:24:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-11-16T20:24:35.137 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-11-16T20:24:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-11-16T20:24:35.138 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-11-16T20:24:35.177 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:35.178 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:35.188 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-11-16T20:24:35.227 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:35.225+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:35.327 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:35.325+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:35.349 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:35.350 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:35.360 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-11-16T20:24:35.428 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:35.428+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:35.479 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:35.480 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:35.490 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-11-16T20:24:35.533 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:35.532+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:35.548 INFO:tasks.rados.rados.0.smithi092.stdout:178: done (1 left) 2020-11-16T20:24:35.548 INFO:tasks.rados.rados.0.smithi092.stdout:184: done (0 left) 2020-11-16T20:24:35.548 INFO:tasks.rados.rados.0.smithi092.stdout:185: snap_create 2020-11-16T20:24:35.631 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:35.628+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:35.649 INFO:teuthology.orchestra.run.smithi013.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:35.650 INFO:teuthology.orchestra.run.smithi013.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:35.660 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-11-16T20:24:35.731 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:35.728+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:35.785 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:35.785 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:35.795 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-11-16T20:24:35.840 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:35.841+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:35.915 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:35.915 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:35.925 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-11-16T20:24:35.932 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:35.928+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:36.033 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:36.033+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:36.083 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:36.083 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:36.093 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-11-16T20:24:36.140 INFO:tasks.ceph.osd.3.smithi092.stderr:2020-11-16T20:24:36.141+0000 7fbeefe02700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 12726) UID: 0 2020-11-16T20:24:36.235 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:36.236+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:36.264 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:36.264 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:36.274 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-11-16T20:24:36.336 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:36.337+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:36.390 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:36.390 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:36.400 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-11-16T20:24:36.437 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:36.436+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:36.538 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:36.536+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:36.555 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2020-11-16T20:24:36.556 INFO:tasks.rados.rados.0.smithi092.stdout:185: done (0 left) 2020-11-16T20:24:36.556 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 33 current snap is 14 2020-11-16T20:24:36.557 INFO:tasks.rados.rados.0.smithi092.stdout:186: seq_num 95 ranges {3039232=794624,3833856=409600,4243456=393216} 2020-11-16T20:24:36.574 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:36.574 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:36.578 INFO:tasks.rados.rados.0.smithi092.stdout:186: writing smithi09216568-33 from 3039232 to 3833856 tid 1 2020-11-16T20:24:36.581 INFO:tasks.rados.rados.0.smithi092.stdout:186: writing smithi09216568-33 from 3833856 to 4243456 tid 2 2020-11-16T20:24:36.584 INFO:tasks.rados.rados.0.smithi092.stdout:186: writing smithi09216568-33 from 4243456 to 4636672 tid 3 2020-11-16T20:24:36.585 INFO:tasks.rados.rados.0.smithi092.stdout:187: rmattr oid 46 current snap is 14 2020-11-16T20:24:36.585 INFO:tasks.rados.rados.0.smithi092.stdout:187: done (1 left) 2020-11-16T20:24:36.585 INFO:tasks.rados.rados.0.smithi092.stdout:188: snap_remove snap 9 2020-11-16T20:24:36.586 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-11-16T20:24:36.671 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:36.673+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:36.739 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:36.737+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:36.743 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:36.743 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:36.754 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-11-16T20:24:36.843 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:36.845+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:36.910 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:36.911 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:36.920 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-11-16T20:24:36.964 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:36.965+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:37.042 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:37.041+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:37.083 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:37.083 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:37.093 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-11-16T20:24:37.151 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-11-16T20:24:37.184 INFO:tasks.ceph.osd.1.smithi092.stderr:2020-11-16T20:24:37.185+0000 7f37d916b700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 12725) UID: 0 2020-11-16T20:24:37.243 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:37.245+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:37.258 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2020-11-16T20:24:37.258 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-11-16T20:24:37.268 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-11-16T20:24:37.340 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2020-11-16T20:24:37.345 INFO:tasks.ceph.osd.4.smithi013.stderr:2020-11-16T20:24:37.344+0000 7f938176c700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 12981) UID: 0 2020-11-16T20:24:37.368 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2020-11-16T20:24:37.396 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2020-11-16T20:24:37.424 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2020-11-16T20:24:37.434 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:37.434 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2020-11-16T20:24:37.434 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2020-11-16T20:24:37.434 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2020-11-16T20:24:37.435 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:37.445 INFO:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-11-16T20:24:37.447 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:37.444+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:37.458 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2020-11-16T20:24:37.488 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2020-11-16T20:24:37.518 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2020-11-16T20:24:37.547 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:37.544+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:37.556 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2020-11-16T20:24:37.558 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-11-16T20:24:37.558 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-11-16T20:24:37.558 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-11-16T20:24:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-11-16T20:24:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-11-16T20:24:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-11-16T20:24:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-11-16T20:24:37.559 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-11-16T20:24:37.567 INFO:tasks.rados.rados.0.smithi092.stdout:188: done (1 left) 2020-11-16T20:24:37.567 INFO:tasks.rados.rados.0.smithi092.stdout:189: read oid 12 snap 7 2020-11-16T20:24:37.567 INFO:tasks.rados.rados.0.smithi092.stdout:189: expect (ObjNum 11 snap 0 seq_num 11) 2020-11-16T20:24:37.567 INFO:tasks.rados.rados.0.smithi092.stdout:190: delete oid 23 current snap is 14 2020-11-16T20:24:37.568 INFO:tasks.rados.rados.0.smithi092.stdout:186: finishing write tid 1 to smithi09216568-33 2020-11-16T20:24:37.568 INFO:tasks.rados.rados.0.smithi092.stdout:186: finishing write tid 2 to smithi09216568-33 2020-11-16T20:24:37.568 INFO:tasks.rados.rados.0.smithi092.stdout:186: finishing write tid 3 to smithi09216568-33 2020-11-16T20:24:37.568 INFO:tasks.rados.rados.0.smithi092.stdout:186: finishing write tid 4 to smithi09216568-33 2020-11-16T20:24:37.568 INFO:tasks.rados.rados.0.smithi092.stdout:186: finishing write tid 5 to smithi09216568-33 2020-11-16T20:24:37.568 INFO:tasks.rados.rados.0.smithi092.stdout:update_object_version oid 33 v 20 (ObjNum 95 snap 14 seq_num 95) dirty exists 2020-11-16T20:24:37.568 INFO:tasks.rados.rados.0.smithi092.stdout:186: left oid 33 (ObjNum 95 snap 14 seq_num 95) 2020-11-16T20:24:37.569 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2020-11-16T20:24:37.573 INFO:tasks.rados.rados.0.smithi092.stdout:189: expect (ObjNum 11 snap 0 seq_num 11) 2020-11-16T20:24:37.582 INFO:tasks.rados.rados.0.smithi092.stdout:186: done (2 left) 2020-11-16T20:24:37.582 INFO:tasks.rados.rados.0.smithi092.stdout:189: done (1 left) 2020-11-16T20:24:37.583 INFO:tasks.rados.rados.0.smithi092.stdout:190: done (0 left) 2020-11-16T20:24:37.583 INFO:tasks.rados.rados.0.smithi092.stdout:191: delete oid 13 current snap is 14 2020-11-16T20:24:37.583 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2020-11-16T20:24:37.583 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2020-11-16T20:24:37.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2020-11-16T20:24:37.583 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2020-11-16T20:24:37.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.584 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:14 3.9s0 3:910fffa8:::smithi09216568-5:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-11-16T20:24:37.584 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:37.584 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.050959836000001, 2020-11-16T20:24:37.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033104548999999997, 2020-11-16T20:24:37.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.585 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.585 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 14 2020-11-16T20:24:37.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.585 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:37.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532266+0000", 2020-11-16T20:24:37.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8910999999999999e-05 2020-11-16T20:24:37.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532325+0000", 2020-11-16T20:24:37.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2323e-05 2020-11-16T20:24:37.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532347+0000", 2020-11-16T20:24:37.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7090999999999999e-05 2020-11-16T20:24:37.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532374+0000", 2020-11-16T20:24:37.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5399999999999998e-06 2020-11-16T20:24:37.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532382+0000", 2020-11-16T20:24:37.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077685499999999999 2020-11-16T20:24:37.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540150+0000", 2020-11-16T20:24:37.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119175 2020-11-16T20:24:37.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540269+0000", 2020-11-16T20:24:37.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011674891 2020-11-16T20:24:37.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551944+0000", 2020-11-16T20:24:37.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010398199999999999 2020-11-16T20:24:37.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.552048+0000", 2020-11-16T20:24:37.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077962600000000002 2020-11-16T20:24:37.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.559845+0000", 2020-11-16T20:24:37.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028470140000000001 2020-11-16T20:24:37.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562692+0000", 2020-11-16T20:24:37.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002587677 2020-11-16T20:24:37.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565279+0000", 2020-11-16T20:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1135000000000001e-05 2020-11-16T20:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565370+0000", 2020-11-16T20:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033104548999999997 2020-11-16T20:24:37.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.595 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.595 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:15 3.9s0 3:910fffa8:::smithi09216568-5:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e30)", 2020-11-16T20:24:37.595 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:37.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.050792239, 2020-11-16T20:24:37.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037521408999999999, 2020-11-16T20:24:37.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:37.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.596 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15 2020-11-16T20:24:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532433+0000", 2020-11-16T20:24:37.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9020000000000001e-05 2020-11-16T20:24:37.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532493+0000", 2020-11-16T20:24:37.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0396e-05 2020-11-16T20:24:37.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532503+0000", 2020-11-16T20:24:37.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9660000000000002e-05 2020-11-16T20:24:37.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532523+0000", 2020-11-16T20:24:37.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8280000000000003e-06 2020-11-16T20:24:37.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532530+0000", 2020-11-16T20:24:37.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077981070000000003 2020-11-16T20:24:37.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540329+0000", 2020-11-16T20:24:37.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8363000000000007e-05 2020-11-16T20:24:37.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540427+0000", 2020-11-16T20:24:37.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019650447000000001 2020-11-16T20:24:37.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560077+0000", 2020-11-16T20:24:37.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2565999999999994e-05 2020-11-16T20:24:37.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560170+0000", 2020-11-16T20:24:37.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052459309999999997 2020-11-16T20:24:37.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565416+0000", 2020-11-16T20:24:37.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1088999999999995e-05 2020-11-16T20:24:37.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565507+0000", 2020-11-16T20:24:37.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044480020000000004 2020-11-16T20:24:37.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.569955+0000", 2020-11-16T20:24:37.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037521408999999999 2020-11-16T20:24:37.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.606 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.606 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:23 3.ds0 3:b099cead:::smithi09216568-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-11-16T20:24:37.606 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:37.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.05034792, 2020-11-16T20:24:37.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025302294999999999, 2020-11-16T20:24:37.607 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.607 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.607 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 23 2020-11-16T20:24:37.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532878+0000", 2020-11-16T20:24:37.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5377999999999997e-05 2020-11-16T20:24:37.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532933+0000", 2020-11-16T20:24:37.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9703000000000001e-05 2020-11-16T20:24:37.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532953+0000", 2020-11-16T20:24:37.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2229e-05 2020-11-16T20:24:37.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.532975+0000", 2020-11-16T20:24:37.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041310599999999998 2020-11-16T20:24:37.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537106+0000", 2020-11-16T20:24:37.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055730839999999999 2020-11-16T20:24:37.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542679+0000", 2020-11-16T20:24:37.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013964700000000001 2020-11-16T20:24:37.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542819+0000", 2020-11-16T20:24:37.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050541550000000003 2020-11-16T20:24:37.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.547873+0000", 2020-11-16T20:24:37.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253999999999993e-05 2020-11-16T20:24:37.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.547934+0000", 2020-11-16T20:24:37.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000519886 2020-11-16T20:24:37.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548454+0000", 2020-11-16T20:24:37.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066483360000000003 2020-11-16T20:24:37.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.555103+0000", 2020-11-16T20:24:37.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026224410000000001 2020-11-16T20:24:37.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.557725+0000", 2020-11-16T20:24:37.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045512200000000002 2020-11-16T20:24:37.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558180+0000", 2020-11-16T20:24:37.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025302294999999999 2020-11-16T20:24:37.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:24 3.ds0 3:b099cead:::smithi09216568-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e30)", 2020-11-16T20:24:37.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:37.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.046053982, 2020-11-16T20:24:37.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025939752999999999, 2020-11-16T20:24:37.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:37.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.619 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.619 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 24 2020-11-16T20:24:37.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.619 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:37.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537172+0000", 2020-11-16T20:24:37.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8302e-05 2020-11-16T20:24:37.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537210+0000", 2020-11-16T20:24:37.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5029999999999997e-06 2020-11-16T20:24:37.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537219+0000", 2020-11-16T20:24:37.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5136e-05 2020-11-16T20:24:37.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537234+0000", 2020-11-16T20:24:37.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2160000000000003e-06 2020-11-16T20:24:37.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.537240+0000", 2020-11-16T20:24:37.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056701290000000003 2020-11-16T20:24:37.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.542910+0000", 2020-11-16T20:24:37.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011685599999999999 2020-11-16T20:24:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543027+0000", 2020-11-16T20:24:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056655689999999996 2020-11-16T20:24:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548692+0000", 2020-11-16T20:24:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9920000000000003e-05 2020-11-16T20:24:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.548742+0000", 2020-11-16T20:24:37.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094867259999999991 2020-11-16T20:24:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558229+0000", 2020-11-16T20:24:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8435000000000003e-05 2020-11-16T20:24:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.558318+0000", 2020-11-16T20:24:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047939610000000002 2020-11-16T20:24:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563111+0000", 2020-11-16T20:24:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025939752999999999 2020-11-16T20:24:37.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:32 3.9s0 3:94376c81:::smithi09216568-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:37.628 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.043663519999999, 2020-11-16T20:24:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026430479999999999, 2020-11-16T20:24:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32 2020-11-16T20:24:37.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539562+0000", 2020-11-16T20:24:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5041999999999999e-05 2020-11-16T20:24:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539617+0000", 2020-11-16T20:24:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6490000000000002e-06 2020-11-16T20:24:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539624+0000", 2020-11-16T20:24:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2020-11-16T20:24:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539636+0000", 2020-11-16T20:24:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.075e-06 2020-11-16T20:24:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539643+0000", 2020-11-16T20:24:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039536490000000001 2020-11-16T20:24:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543597+0000", 2020-11-16T20:24:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112988 2020-11-16T20:24:37.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543710+0000", 2020-11-16T20:24:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016926085 2020-11-16T20:24:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560636+0000", 2020-11-16T20:24:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8800000000000004e-05 2020-11-16T20:24:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560725+0000", 2020-11-16T20:24:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00080697699999999995 2020-11-16T20:24:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.561532+0000", 2020-11-16T20:24:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042868910000000001 2020-11-16T20:24:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565818+0000", 2020-11-16T20:24:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5162999999999998e-05 2020-11-16T20:24:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565904+0000", 2020-11-16T20:24:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9067999999999995e-05 2020-11-16T20:24:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565993+0000", 2020-11-16T20:24:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026430479999999999 2020-11-16T20:24:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:33 3.9s0 3:94376c81:::smithi09216568-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.043541726000001, 2020-11-16T20:24:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030497335, 2020-11-16T20:24:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:37.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33 2020-11-16T20:24:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539684+0000", 2020-11-16T20:24:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8398000000000001e-05 2020-11-16T20:24:37.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539722+0000", 2020-11-16T20:24:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.993e-06 2020-11-16T20:24:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539728+0000", 2020-11-16T20:24:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2140000000000001e-05 2020-11-16T20:24:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539741+0000", 2020-11-16T20:24:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9979999999999997e-06 2020-11-16T20:24:37.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.539745+0000", 2020-11-16T20:24:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004070204 2020-11-16T20:24:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543815+0000", 2020-11-16T20:24:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535999999999999 2020-11-16T20:24:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.543920+0000", 2020-11-16T20:24:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018410834000000001 2020-11-16T20:24:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562331+0000", 2020-11-16T20:24:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4544999999999997e-05 2020-11-16T20:24:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562425+0000", 2020-11-16T20:24:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035976329999999998 2020-11-16T20:24:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.566023+0000", 2020-11-16T20:24:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2955999999999996e-05 2020-11-16T20:24:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.566116+0000", 2020-11-16T20:24:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040652739999999998 2020-11-16T20:24:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.570181+0000", 2020-11-16T20:24:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030497335 2020-11-16T20:24:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:35 3.9s0 3:9cc22e43:::smithi09216568-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:37.650 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.042369179000001, 2020-11-16T20:24:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11108446600000001, 2020-11-16T20:24:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.651 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 35 2020-11-16T20:24:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540857+0000", 2020-11-16T20:24:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4415e-05 2020-11-16T20:24:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540911+0000", 2020-11-16T20:24:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.533e-06 2020-11-16T20:24:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540917+0000", 2020-11-16T20:24:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-05 2020-11-16T20:24:37.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540930+0000", 2020-11-16T20:24:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2769999999999999e-06 2020-11-16T20:24:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540934+0000", 2020-11-16T20:24:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010586527 2020-11-16T20:24:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551521+0000", 2020-11-16T20:24:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012027399999999999 2020-11-16T20:24:37.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.551641+0000", 2020-11-16T20:24:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011083160999999999 2020-11-16T20:24:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562724+0000", 2020-11-16T20:24:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4109000000000004e-05 2020-11-16T20:24:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.562808+0000", 2020-11-16T20:24:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000459751 2020-11-16T20:24:37.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563268+0000", 2020-11-16T20:24:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060973590000000001 2020-11-16T20:24:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.569365+0000", 2020-11-16T20:24:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.082458486999999997 2020-11-16T20:24:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651824+0000", 2020-11-16T20:24:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117173 2020-11-16T20:24:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651941+0000", 2020-11-16T20:24:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11108446600000001 2020-11-16T20:24:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:36 3.9s0 3:9cc22e43:::smithi09216568-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.042250106000001, 2020-11-16T20:24:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.116868257, 2020-11-16T20:24:37.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36 2020-11-16T20:24:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.540976+0000", 2020-11-16T20:24:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9727000000000001e-05 2020-11-16T20:24:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541015+0000", 2020-11-16T20:24:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1739999999999997e-06 2020-11-16T20:24:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541022+0000", 2020-11-16T20:24:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1854e-05 2020-11-16T20:24:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541033+0000", 2020-11-16T20:24:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.799e-06 2020-11-16T20:24:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.541037+0000", 2020-11-16T20:24:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019161628999999999 2020-11-16T20:24:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560199+0000", 2020-11-16T20:24:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102256 2020-11-16T20:24:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.560301+0000", 2020-11-16T20:24:37.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032318479999999998 2020-11-16T20:24:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563533+0000", 2020-11-16T20:24:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2615999999999996e-05 2020-11-16T20:24:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.563626+0000", 2020-11-16T20:24:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088372749 2020-11-16T20:24:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651998+0000", 2020-11-16T20:24:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113002 2020-11-16T20:24:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652111+0000", 2020-11-16T20:24:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057326030000000002 2020-11-16T20:24:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.657844+0000", 2020-11-16T20:24:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.116868257 2020-11-16T20:24:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=10, last_complete=31'6, committed=1, applied=1))", 2020-11-16T20:24:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.017608330000002, 2020-11-16T20:24:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086340027, 2020-11-16T20:24:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565617+0000", 2020-11-16T20:24:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6184999999999999e-05 2020-11-16T20:24:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565664+0000", 2020-11-16T20:24:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7180000000000002e-06 2020-11-16T20:24:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.565672+0000", 2020-11-16T20:24:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085916762999999993 2020-11-16T20:24:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651589+0000", 2020-11-16T20:24:37.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5119999999999997e-06 2020-11-16T20:24:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651596+0000", 2020-11-16T20:24:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3547000000000001e-05 2020-11-16T20:24:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651649+0000", 2020-11-16T20:24:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030830199999999999 2020-11-16T20:24:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.651957+0000", 2020-11-16T20:24:37.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.086340027 2020-11-16T20:24:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:85 3.9s0 3:9bc9395c:::smithi09216568-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:37.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.979509538999999, 2020-11-16T20:24:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048890797999999999, 2020-11-16T20:24:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 85 2020-11-16T20:24:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.603716+0000", 2020-11-16T20:24:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033801700000000003 2020-11-16T20:24:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604054+0000", 2020-11-16T20:24:37.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1904e-05 2020-11-16T20:24:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604066+0000", 2020-11-16T20:24:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2374e-05 2020-11-16T20:24:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604088+0000", 2020-11-16T20:24:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0937e-05 2020-11-16T20:24:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604099+0000", 2020-11-16T20:24:37.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6842999999999996e-05 2020-11-16T20:24:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604176+0000", 2020-11-16T20:24:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180261 2020-11-16T20:24:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604357+0000", 2020-11-16T20:24:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012281449999999999 2020-11-16T20:24:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.605585+0000", 2020-11-16T20:24:37.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074810600000000003 2020-11-16T20:24:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.613066+0000", 2020-11-16T20:24:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039419787999999997 2020-11-16T20:24:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652486+0000", 2020-11-16T20:24:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121469 2020-11-16T20:24:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652607+0000", 2020-11-16T20:24:37.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.048890797999999999 2020-11-16T20:24:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:86 3.9s0 3:9bc9395c:::smithi09216568-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:37.689 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.979067859000001, 2020-11-16T20:24:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071173710000000001, 2020-11-16T20:24:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 86 2020-11-16T20:24:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604158+0000", 2020-11-16T20:24:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7161999999999999e-05 2020-11-16T20:24:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604215+0000", 2020-11-16T20:24:37.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3989999999999998e-06 2020-11-16T20:24:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604222+0000", 2020-11-16T20:24:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3733999999999999e-05 2020-11-16T20:24:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604236+0000", 2020-11-16T20:24:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3090000000000003e-06 2020-11-16T20:24:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604241+0000", 2020-11-16T20:24:37.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074329820000000003 2020-11-16T20:24:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611674+0000", 2020-11-16T20:24:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117886 2020-11-16T20:24:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611792+0000", 2020-11-16T20:24:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040893269000000003 2020-11-16T20:24:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652686+0000", 2020-11-16T20:24:37.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9841999999999998e-05 2020-11-16T20:24:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652775+0000", 2020-11-16T20:24:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052300899999999995 2020-11-16T20:24:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653298+0000", 2020-11-16T20:24:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089837800000000002 2020-11-16T20:24:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.654197+0000", 2020-11-16T20:24:37.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021031319999999999 2020-11-16T20:24:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675228+0000", 2020-11-16T20:24:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010342 2020-11-16T20:24:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675332+0000", 2020-11-16T20:24:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.071173710000000001 2020-11-16T20:24:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:87 3.9s0 3:9bc9395c:::smithi09216568-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.978937031000001, 2020-11-16T20:24:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072859264000000007, 2020-11-16T20:24:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 87 2020-11-16T20:24:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604289+0000", 2020-11-16T20:24:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1205000000000003e-05 2020-11-16T20:24:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604330+0000", 2020-11-16T20:24:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7689999999999999e-06 2020-11-16T20:24:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604337+0000", 2020-11-16T20:24:37.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3181e-05 2020-11-16T20:24:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604350+0000", 2020-11-16T20:24:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1000000000000003e-06 2020-11-16T20:24:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.604355+0000", 2020-11-16T20:24:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074779859999999998 2020-11-16T20:24:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.611833+0000", 2020-11-16T20:24:37.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087081400000000003 2020-11-16T20:24:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.612704+0000", 2020-11-16T20:24:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040880316 2020-11-16T20:24:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653584+0000", 2020-11-16T20:24:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5697000000000003e-05 2020-11-16T20:24:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653670+0000", 2020-11-16T20:24:37.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021708466999999999 2020-11-16T20:24:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675378+0000", 2020-11-16T20:24:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7913999999999998e-05 2020-11-16T20:24:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675476+0000", 2020-11-16T20:24:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016718150000000001 2020-11-16T20:24:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.677148+0000", 2020-11-16T20:24:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.072859264000000007 2020-11-16T20:24:37.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:103 3.9s0 3:98ee58dd:::smithi09216568-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:37.711 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.957298596000001, 2020-11-16T20:24:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037777976999999997, 2020-11-16T20:24:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 103 2020-11-16T20:24:37.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.625927+0000", 2020-11-16T20:24:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000376745 2020-11-16T20:24:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626304+0000", 2020-11-16T20:24:37.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0266e-05 2020-11-16T20:24:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626314+0000", 2020-11-16T20:24:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327e-05 2020-11-16T20:24:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626334+0000", 2020-11-16T20:24:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0482999999999999e-05 2020-11-16T20:24:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626345+0000", 2020-11-16T20:24:37.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7775999999999996e-05 2020-11-16T20:24:37.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626413+0000", 2020-11-16T20:24:37.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020824099999999999 2020-11-16T20:24:37.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626621+0000", 2020-11-16T20:24:37.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097088399999999998 2020-11-16T20:24:37.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.627592+0000", 2020-11-16T20:24:37.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074737739999999999 2020-11-16T20:24:37.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.635066+0000", 2020-11-16T20:24:37.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.028500588 2020-11-16T20:24:37.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663566+0000", 2020-11-16T20:24:37.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013889300000000001 2020-11-16T20:24:37.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663705+0000", 2020-11-16T20:24:37.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037777976999999997 2020-11-16T20:24:37.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.721 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:104 3.9s0 3:98ee58dd:::smithi09216568-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:37.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:37.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.956824906000001, 2020-11-16T20:24:37.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109118772, 2020-11-16T20:24:37.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 104 2020-11-16T20:24:37.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.723 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:37.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626401+0000", 2020-11-16T20:24:37.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4968999999999997e-05 2020-11-16T20:24:37.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626436+0000", 2020-11-16T20:24:37.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.33e-06 2020-11-16T20:24:37.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626440+0000", 2020-11-16T20:24:37.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1780000000000003e-06 2020-11-16T20:24:37.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626448+0000", 2020-11-16T20:24:37.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5559999999999998e-06 2020-11-16T20:24:37.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626454+0000", 2020-11-16T20:24:37.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082374119999999995 2020-11-16T20:24:37.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634691+0000", 2020-11-16T20:24:37.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6558999999999995e-05 2020-11-16T20:24:37.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634788+0000", 2020-11-16T20:24:37.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029028163999999999 2020-11-16T20:24:37.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663816+0000", 2020-11-16T20:24:37.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9767999999999998e-05 2020-11-16T20:24:37.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.663906+0000", 2020-11-16T20:24:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088046700000000001 2020-11-16T20:24:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.664786+0000", 2020-11-16T20:24:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001328255 2020-11-16T20:24:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.666114+0000", 2020-11-16T20:24:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069334706999999995 2020-11-16T20:24:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735449+0000", 2020-11-16T20:24:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0407000000000005e-05 2020-11-16T20:24:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735520+0000", 2020-11-16T20:24:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.109118772 2020-11-16T20:24:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:105 3.9s0 3:98ee58dd:::smithi09216568-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.956725716000001, 2020-11-16T20:24:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11144889600000001, 2020-11-16T20:24:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 105 2020-11-16T20:24:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626500+0000", 2020-11-16T20:24:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0299000000000002e-05 2020-11-16T20:24:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626540+0000", 2020-11-16T20:24:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3050000000000004e-06 2020-11-16T20:24:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626548+0000", 2020-11-16T20:24:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.878e-05 2020-11-16T20:24:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626566+0000", 2020-11-16T20:24:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3839999999999996e-06 2020-11-16T20:24:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.626572+0000", 2020-11-16T20:24:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082488189999999992 2020-11-16T20:24:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634821+0000", 2020-11-16T20:24:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4552e-05 2020-11-16T20:24:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.634905+0000", 2020-11-16T20:24:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.030415658000000002 2020-11-16T20:24:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.665321+0000", 2020-11-16T20:24:37.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0865000000000005e-05 2020-11-16T20:24:37.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.665412+0000", 2020-11-16T20:24:37.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.070142494 2020-11-16T20:24:37.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735554+0000", 2020-11-16T20:24:37.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9005000000000007e-05 2020-11-16T20:24:37.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735633+0000", 2020-11-16T20:24:37.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023157350000000002 2020-11-16T20:24:37.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.737949+0000", 2020-11-16T20:24:37.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11144889600000001 2020-11-16T20:24:37.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:108 3.as0 3:5d07813d:::smithi09216568-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.952297478999999, 2020-11-16T20:24:37.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024149034999999999, 2020-11-16T20:24:37.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:37.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 108 2020-11-16T20:24:37.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:37.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630928+0000", 2020-11-16T20:24:37.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4607999999999997e-05 2020-11-16T20:24:37.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630973+0000", 2020-11-16T20:24:37.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4680000000000003e-06 2020-11-16T20:24:37.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.630980+0000", 2020-11-16T20:24:37.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019917399999999999 2020-11-16T20:24:37.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631179+0000", 2020-11-16T20:24:37.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4740000000000002e-06 2020-11-16T20:24:37.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.631186+0000", 2020-11-16T20:24:37.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010229520000000001 2020-11-16T20:24:37.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641415+0000", 2020-11-16T20:24:37.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3351999999999997e-05 2020-11-16T20:24:37.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.641499+0000", 2020-11-16T20:24:37.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0074322040000000004 2020-11-16T20:24:37.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.648931+0000", 2020-11-16T20:24:37.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.166e-05 2020-11-16T20:24:37.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.649023+0000", 2020-11-16T20:24:37.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039121529999999998 2020-11-16T20:24:37.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.652935+0000", 2020-11-16T20:24:37.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140617 2020-11-16T20:24:37.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.653075+0000", 2020-11-16T20:24:37.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020018050000000002 2020-11-16T20:24:37.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.655077+0000", 2020-11-16T20:24:37.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024149034999999999 2020-11-16T20:24:37.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.754 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=24, last_complete=31'10, committed=1, applied=1))", 2020-11-16T20:24:37.754 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:37.754 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.908240739, 2020-11-16T20:24:37.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060547953000000002, 2020-11-16T20:24:37.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:37.755 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:37.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.674985+0000", 2020-11-16T20:24:37.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6891000000000003e-05 2020-11-16T20:24:37.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675022+0000", 2020-11-16T20:24:37.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3489999999999998e-06 2020-11-16T20:24:37.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.675031+0000", 2020-11-16T20:24:37.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060268058999999999 2020-11-16T20:24:37.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735299+0000", 2020-11-16T20:24:37.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7539999999999995e-06 2020-11-16T20:24:37.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735308+0000", 2020-11-16T20:24:37.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7015999999999998e-05 2020-11-16T20:24:37.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735355+0000", 2020-11-16T20:24:37.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017788400000000001 2020-11-16T20:24:37.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.735533+0000", 2020-11-16T20:24:37.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060547953000000002 2020-11-16T20:24:37.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.760 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.760 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:153 3.9s0 3:94376c81:::smithi09216568-11:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:37.761 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:37.761 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.872435984999999, 2020-11-16T20:24:37.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056909992999999999, 2020-11-16T20:24:37.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:37.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.762 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 153 2020-11-16T20:24:37.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.762 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:37.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710790+0000", 2020-11-16T20:24:37.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6903999999999998e-05 2020-11-16T20:24:37.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710827+0000", 2020-11-16T20:24:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0011e-05 2020-11-16T20:24:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710837+0000", 2020-11-16T20:24:37.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4769e-05 2020-11-16T20:24:37.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710851+0000", 2020-11-16T20:24:37.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1129999999999997e-06 2020-11-16T20:24:37.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710858+0000", 2020-11-16T20:24:37.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6598999999999998e-05 2020-11-16T20:24:37.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710914+0000", 2020-11-16T20:24:37.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110771 2020-11-16T20:24:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711025+0000", 2020-11-16T20:24:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011061122 2020-11-16T20:24:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2020-11-16T20:24:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.722086+0000", 2020-11-16T20:24:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055101880000000001 2020-11-16T20:24:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2020-11-16T20:24:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.727596+0000", 2020-11-16T20:24:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039941002000000003 2020-11-16T20:24:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2020-11-16T20:24:37.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767537+0000", 2020-11-16T20:24:37.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000162514 2020-11-16T20:24:37.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767700+0000", 2020-11-16T20:24:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056909992999999999 2020-11-16T20:24:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4444.0:154 3.9s0 3:94376c81:::smithi09216568-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:37.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:37.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.872316520999998, 2020-11-16T20:24:37.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058295893000000001, 2020-11-16T20:24:37.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2020-11-16T20:24:37.771 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2020-11-16T20:24:37.771 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4444", 2020-11-16T20:24:37.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:37.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2020-11-16T20:24:37.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.772 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:37.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710909+0000", 2020-11-16T20:24:37.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7756999999999998e-05 2020-11-16T20:24:37.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710947+0000", 2020-11-16T20:24:37.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7429999999999996e-06 2020-11-16T20:24:37.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710954+0000", 2020-11-16T20:24:37.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2998e-05 2020-11-16T20:24:37.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710967+0000", 2020-11-16T20:24:37.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8250000000000004e-06 2020-11-16T20:24:37.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.710972+0000", 2020-11-16T20:24:37.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022201499999999999 2020-11-16T20:24:37.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711194+0000", 2020-11-16T20:24:37.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1608e-05 2020-11-16T20:24:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2020-11-16T20:24:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.711225+0000", 2020-11-16T20:24:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012795857000000001 2020-11-16T20:24:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724021+0000", 2020-11-16T20:24:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7220999999999998e-05 2020-11-16T20:24:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724098+0000", 2020-11-16T20:24:37.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044522785000000002 2020-11-16T20:24:37.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768621+0000", 2020-11-16T20:24:37.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026034400000000003 2020-11-16T20:24:37.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2020-11-16T20:24:37.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768881+0000", 2020-11-16T20:24:37.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032373999999999999 2020-11-16T20:24:37.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.769205+0000", 2020-11-16T20:24:37.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058295893000000001 2020-11-16T20:24:37.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.781 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.781 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 31/23 ECSubWriteReply(tid=26, last_complete=31'11, committed=1, applied=1))", 2020-11-16T20:24:37.781 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:37.781 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.858876819999999, 2020-11-16T20:24:37.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044143454999999998, 2020-11-16T20:24:37.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2020-11-16T20:24:37.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2020-11-16T20:24:37.782 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2020-11-16T20:24:37.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2020-11-16T20:24:37.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:37.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2020-11-16T20:24:37.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2020-11-16T20:24:37.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724349+0000", 2020-11-16T20:24:37.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7308999999999999e-05 2020-11-16T20:24:37.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2020-11-16T20:24:37.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724376+0000", 2020-11-16T20:24:37.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9070000000000004e-06 2020-11-16T20:24:37.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2020-11-16T20:24:37.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.724382+0000", 2020-11-16T20:24:37.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042942860999999999 2020-11-16T20:24:37.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2020-11-16T20:24:37.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767325+0000", 2020-11-16T20:24:37.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2480000000000001e-05 2020-11-16T20:24:37.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2020-11-16T20:24:37.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767337+0000", 2020-11-16T20:24:37.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.177e-05 2020-11-16T20:24:37.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2020-11-16T20:24:37.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.767399+0000", 2020-11-16T20:24:37.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010931280000000001 2020-11-16T20:24:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2020-11-16T20:24:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2020-11-16T20:24:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2020-11-16T20:24:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2020-11-16T20:24:15.768492+0000", 2020-11-16T20:24:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044143454999999998 2020-11-16T20:24:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2020-11-16T20:24:37.788 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2020-11-16T20:24:37.788 INFO:teuthology.orchestra.run.smithi092.stdout:} 2020-11-16T20:24:37.789 INFO:tasks.rados.rados.0.smithi092.stdout:191: done (0 left) 2020-11-16T20:24:37.790 INFO:tasks.rados.rados.0.smithi092.stdout:192: read oid 24 snap -1 2020-11-16T20:24:37.790 INFO:tasks.rados.rados.0.smithi092.stdout:192: expect (ObjNum 66 snap 5 seq_num 66) 2020-11-16T20:24:37.790 INFO:tasks.rados.rados.0.smithi092.stdout:193: rmattr oid 29 current snap is 14 2020-11-16T20:24:37.790 INFO:tasks.rados.rados.0.smithi092.stdout:193: done (1 left) 2020-11-16T20:24:37.790 INFO:tasks.rados.rados.0.smithi092.stdout:194: delete oid 13 current snap is 14 2020-11-16T20:24:37.790 INFO:tasks.rados.rados.0.smithi092.stdout:194: done (1 left) 2020-11-16T20:24:37.790 INFO:tasks.rados.rados.0.smithi092.stdout:195: copy_from oid 15 from oid 38 current snap is 14 2020-11-16T20:24:37.791 INFO:tasks.rados.rados.0.smithi092.stdout:196: snap_remove snap 5 2020-11-16T20:24:37.791 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-11-16T20:24:37.792 INFO:tasks.ceph.osd.7.smithi013.stderr:2020-11-16T20:24:37.788+0000 7f11f5aaa700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 12986) UID: 0 2020-11-16T20:24:37.890 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:37.889+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:37.925 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:37.925 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:37.926 INFO:teuthology.orchestra.run.smithi013.stdout: "num_ops": 0 2020-11-16T20:24:37.926 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:37.936 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-11-16T20:24:37.991 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:37.989+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:38.060 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:38.061 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:38.061 INFO:teuthology.orchestra.run.smithi013.stdout: "complaint_time": 30, 2020-11-16T20:24:38.061 INFO:teuthology.orchestra.run.smithi013.stdout: "num_blocked_ops": 0 2020-11-16T20:24:38.061 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:38.071 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-11-16T20:24:38.091 INFO:tasks.ceph.osd.0.smithi092.stderr:2020-11-16T20:24:38.089+0000 7ff23e8bd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 12723) UID: 0 2020-11-16T20:24:38.192 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:38.193+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:38.193 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:38.194 INFO:teuthology.orchestra.run.smithi013.stdout: "size": 20, 2020-11-16T20:24:38.194 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 600, 2020-11-16T20:24:38.194 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [ 2020-11-16T20:24:38.194 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.194 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:57 3.8s0 3:159ef6eb:::smithi09216568-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:38.195 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.570222+0000", 2020-11-16T20:24:38.195 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.623734836000001, 2020-11-16T20:24:38.195 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.041859702999999998, 2020-11-16T20:24:38.195 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.195 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:38.195 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.195 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.196 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.196 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 57 2020-11-16T20:24:38.196 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.196 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.196 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.196 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.196 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570222+0000", 2020-11-16T20:24:38.197 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.197 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.197 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.197 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.197 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570222+0000", 2020-11-16T20:24:38.197 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.0971999999999998e-05 2020-11-16T20:24:38.197 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.198 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.198 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.198 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570243+0000", 2020-11-16T20:24:38.198 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.301e-06 2020-11-16T20:24:38.198 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.198 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.199 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.199 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570247+0000", 2020-11-16T20:24:38.199 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.5560000000000002e-06 2020-11-16T20:24:38.199 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.199 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.199 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.199 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570254+0000", 2020-11-16T20:24:38.200 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.2610000000000002e-06 2020-11-16T20:24:38.200 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.200 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.200 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.200 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.570258+0000", 2020-11-16T20:24:38.200 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0053828970000000002 2020-11-16T20:24:38.200 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.201 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.201 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.201 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.575641+0000", 2020-11-16T20:24:38.201 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000103824 2020-11-16T20:24:38.201 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.201 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.201 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.202 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.575744+0000", 2020-11-16T20:24:38.202 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.008211444 2020-11-16T20:24:38.202 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.202 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.202 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.202 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.583956+0000", 2020-11-16T20:24:38.202 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000105116 2020-11-16T20:24:38.203 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.203 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.203 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.203 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.584061+0000", 2020-11-16T20:24:38.203 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.015111940000000001 2020-11-16T20:24:38.203 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.203 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.204 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.204 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599173+0000", 2020-11-16T20:24:38.204 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.3417999999999995e-05 2020-11-16T20:24:38.204 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.204 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.204 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.204 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599236+0000", 2020-11-16T20:24:38.205 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012844974 2020-11-16T20:24:38.205 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.205 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.205 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.205 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612081+0000", 2020-11-16T20:24:38.205 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.041859702999999998 2020-11-16T20:24:38.205 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.206 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.206 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.206 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.206 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.206 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:74 3.8s0 3:1ae67b5f:::smithi09216568-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:38.206 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.587001+0000", 2020-11-16T20:24:38.206 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.606955555999999, 2020-11-16T20:24:38.207 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.048715451999999999, 2020-11-16T20:24:38.207 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.207 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.207 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.207 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.207 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.207 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 74 2020-11-16T20:24:38.208 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.208 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.208 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.208 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.208 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587001+0000", 2020-11-16T20:24:38.208 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.208 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.209 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.209 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.209 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587001+0000", 2020-11-16T20:24:38.209 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.4703999999999997e-05 2020-11-16T20:24:38.209 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.209 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.209 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.210 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587056+0000", 2020-11-16T20:24:38.210 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.6519999999999995e-06 2020-11-16T20:24:38.210 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.210 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.210 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.210 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587064+0000", 2020-11-16T20:24:38.210 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.0737000000000001e-05 2020-11-16T20:24:38.211 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.211 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.211 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.211 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587075+0000", 2020-11-16T20:24:38.211 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.0350000000000003e-06 2020-11-16T20:24:38.211 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.211 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.212 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.212 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587079+0000", 2020-11-16T20:24:38.212 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0062840250000000004 2020-11-16T20:24:38.212 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.212 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.212 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.212 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.593363+0000", 2020-11-16T20:24:38.213 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000125931 2020-11-16T20:24:38.213 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.213 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.213 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.213 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.593489+0000", 2020-11-16T20:24:38.213 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017552346999999999 2020-11-16T20:24:38.213 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.214 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.214 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.214 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611041+0000", 2020-11-16T20:24:38.214 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000105302 2020-11-16T20:24:38.214 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.214 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.214 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.215 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611147+0000", 2020-11-16T20:24:38.215 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00032716199999999998 2020-11-16T20:24:38.215 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.215 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.215 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.215 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611474+0000", 2020-11-16T20:24:38.215 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0036760360000000001 2020-11-16T20:24:38.216 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.216 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.216 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.216 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.615150+0000", 2020-11-16T20:24:38.216 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.020462314999999998 2020-11-16T20:24:38.216 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.216 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.217 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.217 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635612+0000", 2020-11-16T20:24:38.217 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000104206 2020-11-16T20:24:38.217 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.217 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.217 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.217 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635716+0000", 2020-11-16T20:24:38.218 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.048715451999999999 2020-11-16T20:24:38.218 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.218 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.218 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.218 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.218 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.218 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:75 3.8s0 3:1ae67b5f:::smithi09216568-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:38.219 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.587131+0000", 2020-11-16T20:24:38.219 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.606825870000002, 2020-11-16T20:24:38.219 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.050649959000000001, 2020-11-16T20:24:38.219 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.219 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:38.219 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.219 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.220 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.220 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 75 2020-11-16T20:24:38.220 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.220 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.220 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.220 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.220 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587131+0000", 2020-11-16T20:24:38.221 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.221 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.221 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.221 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.221 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587131+0000", 2020-11-16T20:24:38.221 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.2969000000000001e-05 2020-11-16T20:24:38.221 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.222 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.222 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.222 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587174+0000", 2020-11-16T20:24:38.222 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.9169999999999995e-06 2020-11-16T20:24:38.222 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.222 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.222 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.223 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587183+0000", 2020-11-16T20:24:38.223 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4047e-05 2020-11-16T20:24:38.223 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.223 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.223 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.223 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587197+0000", 2020-11-16T20:24:38.223 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.2350000000000002e-06 2020-11-16T20:24:38.224 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.224 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.224 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.224 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.587204+0000", 2020-11-16T20:24:38.224 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0064723080000000004 2020-11-16T20:24:38.224 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.224 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.225 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.225 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.593676+0000", 2020-11-16T20:24:38.225 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00014109800000000001 2020-11-16T20:24:38.225 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.225 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.225 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.225 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.593817+0000", 2020-11-16T20:24:38.226 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017874455000000001 2020-11-16T20:24:38.226 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.226 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.226 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.226 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611692+0000", 2020-11-16T20:24:38.226 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.5184000000000002e-05 2020-11-16T20:24:38.226 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.227 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.227 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.227 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.611787+0000", 2020-11-16T20:24:38.227 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.024003231999999999 2020-11-16T20:24:38.227 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.227 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.227 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.228 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635790+0000", 2020-11-16T20:24:38.228 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000107527 2020-11-16T20:24:38.228 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.228 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.228 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.228 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.635898+0000", 2020-11-16T20:24:38.228 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001882987 2020-11-16T20:24:38.229 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.229 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.229 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.229 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.637781+0000", 2020-11-16T20:24:38.229 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.050649959000000001 2020-11-16T20:24:38.229 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.229 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.230 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.230 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.230 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.230 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:80 3.6s0 3:6ded7fc6:::smithi09216568-27:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:38.230 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.598238+0000", 2020-11-16T20:24:38.230 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.595718325, 2020-11-16T20:24:38.230 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.035012275000000002, 2020-11-16T20:24:38.231 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.231 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.231 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.231 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.231 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.231 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 80 2020-11-16T20:24:38.231 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.232 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.232 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.232 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.232 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598238+0000", 2020-11-16T20:24:38.232 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.232 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.232 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.232 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.233 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598238+0000", 2020-11-16T20:24:38.233 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.9252000000000001e-05 2020-11-16T20:24:38.233 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.233 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.233 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.233 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598297+0000", 2020-11-16T20:24:38.233 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.9910000000000002e-06 2020-11-16T20:24:38.234 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.234 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.234 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.234 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598305+0000", 2020-11-16T20:24:38.234 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4090999999999999e-05 2020-11-16T20:24:38.234 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.235 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.235 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.235 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598319+0000", 2020-11-16T20:24:38.235 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.2589999999999997e-06 2020-11-16T20:24:38.235 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.235 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.235 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.235 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598326+0000", 2020-11-16T20:24:38.236 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.006674915 2020-11-16T20:24:38.236 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.236 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.236 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.236 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.605001+0000", 2020-11-16T20:24:38.236 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000152025 2020-11-16T20:24:38.236 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.237 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.237 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.237 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.605153+0000", 2020-11-16T20:24:38.237 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013891341 2020-11-16T20:24:38.237 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.237 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.237 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.238 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619044+0000", 2020-11-16T20:24:38.238 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00010469 2020-11-16T20:24:38.238 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.238 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.238 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.238 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619149+0000", 2020-11-16T20:24:38.238 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00041109499999999999 2020-11-16T20:24:38.239 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.239 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.239 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.239 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619560+0000", 2020-11-16T20:24:38.239 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00083761899999999995 2020-11-16T20:24:38.239 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.239 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.240 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.240 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.620397+0000", 2020-11-16T20:24:38.240 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012771629 2020-11-16T20:24:38.240 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.240 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.240 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.240 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.633169+0000", 2020-11-16T20:24:38.241 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.1367999999999998e-05 2020-11-16T20:24:38.241 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.241 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.241 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.241 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.633250+0000", 2020-11-16T20:24:38.241 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.035012275000000002 2020-11-16T20:24:38.241 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.242 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.242 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.242 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.242 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.242 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:81 3.6s0 3:6ded7fc6:::smithi09216568-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:38.242 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.598374+0000", 2020-11-16T20:24:38.242 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.595582873000001, 2020-11-16T20:24:38.243 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036153288999999998, 2020-11-16T20:24:38.243 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.243 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:38.243 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.243 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.243 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.243 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 81 2020-11-16T20:24:38.244 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.244 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.244 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.244 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.244 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598374+0000", 2020-11-16T20:24:38.244 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.244 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.245 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.245 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.245 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598374+0000", 2020-11-16T20:24:38.245 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.3807000000000001e-05 2020-11-16T20:24:38.245 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.245 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.245 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.246 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598397+0000", 2020-11-16T20:24:38.246 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.3869999999999997e-06 2020-11-16T20:24:38.246 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.246 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.246 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.246 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598402+0000", 2020-11-16T20:24:38.246 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.2539999999999992e-06 2020-11-16T20:24:38.247 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.247 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.247 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.247 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598410+0000", 2020-11-16T20:24:38.247 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.5009999999999998e-06 2020-11-16T20:24:38.247 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.247 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.248 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.248 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598415+0000", 2020-11-16T20:24:38.248 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0067833110000000002 2020-11-16T20:24:38.248 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.248 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.248 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.248 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.605198+0000", 2020-11-16T20:24:38.249 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0051836130000000001 2020-11-16T20:24:38.249 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.249 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.249 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.249 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.610381+0000", 2020-11-16T20:24:38.249 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0093756299999999994 2020-11-16T20:24:38.249 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.250 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.250 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.250 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619757+0000", 2020-11-16T20:24:38.250 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.0941999999999998e-05 2020-11-16T20:24:38.250 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.250 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.250 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.251 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.619848+0000", 2020-11-16T20:24:38.251 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013541793999999999 2020-11-16T20:24:38.251 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.251 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.251 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.251 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.633390+0000", 2020-11-16T20:24:38.251 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00010958700000000001 2020-11-16T20:24:38.252 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.252 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.252 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.252 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.633499+0000", 2020-11-16T20:24:38.252 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0010274629999999999 2020-11-16T20:24:38.252 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.252 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.253 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.253 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.634527+0000", 2020-11-16T20:24:38.253 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036153288999999998 2020-11-16T20:24:38.253 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.253 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.253 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.253 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.254 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.254 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:82 3.8s0 3:18af2eec:::smithi09216568-28:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:38.254 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.598442+0000", 2020-11-16T20:24:38.254 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.595514205000001, 2020-11-16T20:24:38.254 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030589345, 2020-11-16T20:24:38.254 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.254 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.255 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.255 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.255 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.255 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 82 2020-11-16T20:24:38.255 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.255 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.255 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.256 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.256 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598442+0000", 2020-11-16T20:24:38.256 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.256 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.256 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.256 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.256 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598442+0000", 2020-11-16T20:24:38.257 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00048240000000000002 2020-11-16T20:24:38.257 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.257 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.257 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.257 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598925+0000", 2020-11-16T20:24:38.257 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.8969999999999996e-06 2020-11-16T20:24:38.257 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.258 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.258 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.258 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598931+0000", 2020-11-16T20:24:38.258 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.8952000000000001e-05 2020-11-16T20:24:38.258 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.258 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.258 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.259 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598950+0000", 2020-11-16T20:24:38.259 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.9427000000000001e-05 2020-11-16T20:24:38.259 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.259 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.259 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.259 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.598979+0000", 2020-11-16T20:24:38.259 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00048842999999999998 2020-11-16T20:24:38.260 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.260 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.260 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.260 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599467+0000", 2020-11-16T20:24:38.260 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000186918 2020-11-16T20:24:38.260 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.260 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.261 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.261 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599654+0000", 2020-11-16T20:24:38.261 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0058116189999999996 2020-11-16T20:24:38.261 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.261 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.261 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.261 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.605466+0000", 2020-11-16T20:24:38.262 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0094244389999999997 2020-11-16T20:24:38.262 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.262 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.262 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.262 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.614890+0000", 2020-11-16T20:24:38.262 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013836742000000001 2020-11-16T20:24:38.262 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.263 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.263 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.263 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.628727+0000", 2020-11-16T20:24:38.263 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00030452099999999998 2020-11-16T20:24:38.263 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.263 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.263 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.264 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.629032+0000", 2020-11-16T20:24:38.264 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030589345 2020-11-16T20:24:38.264 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.264 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.264 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.264 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.264 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.265 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:83 3.8s0 3:18af2eec:::smithi09216568-28:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:38.265 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.599013+0000", 2020-11-16T20:24:38.265 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.594943081, 2020-11-16T20:24:38.265 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.037367409999999997, 2020-11-16T20:24:38.265 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.265 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.265 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.266 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.266 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.266 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 83 2020-11-16T20:24:38.266 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.266 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.266 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.266 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.267 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599013+0000", 2020-11-16T20:24:38.267 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.267 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.267 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.267 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.267 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599013+0000", 2020-11-16T20:24:38.267 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.1021e-05 2020-11-16T20:24:38.268 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.268 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.268 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.268 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599044+0000", 2020-11-16T20:24:38.268 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.2189999999999998e-06 2020-11-16T20:24:38.268 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.268 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.269 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.269 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599049+0000", 2020-11-16T20:24:38.269 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.238e-06 2020-11-16T20:24:38.269 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.269 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.269 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.269 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599056+0000", 2020-11-16T20:24:38.270 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.2409999999999997e-06 2020-11-16T20:24:38.270 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.270 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.270 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.270 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599060+0000", 2020-11-16T20:24:38.270 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013073023 2020-11-16T20:24:38.270 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.271 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.271 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.271 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612133+0000", 2020-11-16T20:24:38.271 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00010226999999999999 2020-11-16T20:24:38.271 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.271 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.271 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.272 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612235+0000", 2020-11-16T20:24:38.272 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.016896411 2020-11-16T20:24:38.272 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.272 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.272 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.272 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.629132+0000", 2020-11-16T20:24:38.272 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.9254999999999998e-05 2020-11-16T20:24:38.273 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.273 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.273 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.273 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.629231+0000", 2020-11-16T20:24:38.273 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00051924100000000004 2020-11-16T20:24:38.273 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.273 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.274 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.274 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.629750+0000", 2020-11-16T20:24:38.274 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001168388 2020-11-16T20:24:38.274 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.274 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.274 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.274 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.630919+0000", 2020-11-16T20:24:38.275 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0053587779999999998 2020-11-16T20:24:38.275 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.275 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.275 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.275 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.636277+0000", 2020-11-16T20:24:38.275 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000103325 2020-11-16T20:24:38.275 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.276 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.276 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.276 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.636381+0000", 2020-11-16T20:24:38.276 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.037367409999999997 2020-11-16T20:24:38.276 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.276 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.276 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.277 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.277 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.277 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:84 3.8s0 3:18af2eec:::smithi09216568-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:38.277 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.599104+0000", 2020-11-16T20:24:38.277 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.594852420999999, 2020-11-16T20:24:38.277 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.038956530000000003, 2020-11-16T20:24:38.277 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.278 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:38.278 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.278 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.278 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.278 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 84 2020-11-16T20:24:38.278 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.278 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.279 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.279 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.279 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599104+0000", 2020-11-16T20:24:38.279 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.279 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.279 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.279 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.280 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599104+0000", 2020-11-16T20:24:38.280 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.8121000000000001e-05 2020-11-16T20:24:38.280 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.280 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.280 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.280 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599132+0000", 2020-11-16T20:24:38.280 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.9519999999999996e-06 2020-11-16T20:24:38.281 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.281 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.281 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.281 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599136+0000", 2020-11-16T20:24:38.281 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.6709999999999999e-06 2020-11-16T20:24:38.281 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.281 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.282 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.282 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599144+0000", 2020-11-16T20:24:38.282 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.5540000000000002e-06 2020-11-16T20:24:38.282 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.282 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.282 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.282 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.599147+0000", 2020-11-16T20:24:38.283 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013124421000000001 2020-11-16T20:24:38.283 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.283 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.283 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.283 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612272+0000", 2020-11-16T20:24:38.283 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.2768000000000006e-05 2020-11-16T20:24:38.283 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.284 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.284 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.284 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.612335+0000", 2020-11-16T20:24:38.284 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017973705 2020-11-16T20:24:38.284 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.284 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.284 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.285 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.630308+0000", 2020-11-16T20:24:38.285 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.1384999999999994e-05 2020-11-16T20:24:38.285 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.285 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.285 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.285 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.630390+0000", 2020-11-16T20:24:38.285 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0063989729999999996 2020-11-16T20:24:38.286 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.286 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.286 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.286 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.636789+0000", 2020-11-16T20:24:38.286 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00013566300000000001 2020-11-16T20:24:38.286 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.286 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.287 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.287 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.636924+0000", 2020-11-16T20:24:38.287 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001136317 2020-11-16T20:24:38.287 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.287 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.287 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.287 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.638061+0000", 2020-11-16T20:24:38.288 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.038956530000000003 2020-11-16T20:24:38.288 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.288 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.288 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.288 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.288 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.288 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:129 3.2s0 3:414e2876:::smithi09216568-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:38.289 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.671721+0000", 2020-11-16T20:24:38.289 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.522235772999998, 2020-11-16T20:24:38.289 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.041126508999999999, 2020-11-16T20:24:38.289 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.289 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:38.289 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.289 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.290 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.290 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 129 2020-11-16T20:24:38.290 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.290 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.290 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.290 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.290 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671721+0000", 2020-11-16T20:24:38.291 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.291 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.291 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.291 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.291 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671721+0000", 2020-11-16T20:24:38.291 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.1564999999999999e-05 2020-11-16T20:24:38.291 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.292 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.292 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.292 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671762+0000", 2020-11-16T20:24:38.292 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.2339999999999997e-06 2020-11-16T20:24:38.292 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.292 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.292 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.293 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671770+0000", 2020-11-16T20:24:38.293 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.501e-05 2020-11-16T20:24:38.293 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.293 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.293 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.293 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671785+0000", 2020-11-16T20:24:38.293 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.6481e-05 2020-11-16T20:24:38.294 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.294 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.294 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.294 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.671801+0000", 2020-11-16T20:24:38.294 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0064728160000000002 2020-11-16T20:24:38.294 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.294 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.295 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.295 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.678274+0000", 2020-11-16T20:24:38.295 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.0265e-05 2020-11-16T20:24:38.295 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.295 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.295 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.295 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.678334+0000", 2020-11-16T20:24:38.296 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017875683 2020-11-16T20:24:38.296 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.296 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.296 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.296 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.696210+0000", 2020-11-16T20:24:38.296 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00011280700000000001 2020-11-16T20:24:38.296 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.297 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.297 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.297 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.696323+0000", 2020-11-16T20:24:38.297 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0042602059999999999 2020-11-16T20:24:38.297 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.297 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.297 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.298 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700583+0000", 2020-11-16T20:24:38.298 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.6906999999999999e-05 2020-11-16T20:24:38.298 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.298 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.298 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.298 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700680+0000", 2020-11-16T20:24:38.298 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012167535 2020-11-16T20:24:38.299 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.299 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.299 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.299 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.712847+0000", 2020-11-16T20:24:38.299 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.041126508999999999 2020-11-16T20:24:38.299 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.299 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.300 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.300 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.300 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.300 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:131 3.2s0 3:47d6be26:::smithi09216568-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-11-16T20:24:38.300 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.674850+0000", 2020-11-16T20:24:38.300 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.519106197999999, 2020-11-16T20:24:38.300 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036389118999999998, 2020-11-16T20:24:38.301 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.301 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.301 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.301 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.301 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.301 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 131 2020-11-16T20:24:38.301 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.302 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.302 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.302 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.302 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674850+0000", 2020-11-16T20:24:38.302 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.302 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.302 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.303 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.303 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674850+0000", 2020-11-16T20:24:38.303 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.0031000000000001e-05 2020-11-16T20:24:38.303 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.303 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.303 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.303 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674880+0000", 2020-11-16T20:24:38.303 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.6959999999999998e-06 2020-11-16T20:24:38.304 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.304 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.304 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.304 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674884+0000", 2020-11-16T20:24:38.304 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.9369999999999998e-06 2020-11-16T20:24:38.304 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.305 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.305 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.305 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674891+0000", 2020-11-16T20:24:38.305 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.7549999999999999e-06 2020-11-16T20:24:38.305 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.305 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.305 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.305 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674894+0000", 2020-11-16T20:24:38.306 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012029560999999999 2020-11-16T20:24:38.306 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.306 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.306 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.306 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.686923+0000", 2020-11-16T20:24:38.306 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.6857e-05 2020-11-16T20:24:38.307 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.307 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.307 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.307 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.686990+0000", 2020-11-16T20:24:38.307 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.011909622999999999 2020-11-16T20:24:38.307 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.307 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.308 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.308 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.698900+0000", 2020-11-16T20:24:38.308 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000178133 2020-11-16T20:24:38.308 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.308 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.308 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.308 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.699078+0000", 2020-11-16T20:24:38.309 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00066865499999999999 2020-11-16T20:24:38.309 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.309 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.309 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.309 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.699747+0000", 2020-11-16T20:24:38.309 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0013938620000000001 2020-11-16T20:24:38.309 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.309 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.310 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.310 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.701140+0000", 2020-11-16T20:24:38.310 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0060941889999999999 2020-11-16T20:24:38.310 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.310 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.310 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.310 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.707235+0000", 2020-11-16T20:24:38.311 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0040048200000000001 2020-11-16T20:24:38.311 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.311 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.311 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.311 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.711239+0000", 2020-11-16T20:24:38.311 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036389118999999998 2020-11-16T20:24:38.311 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.312 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.312 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.312 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.312 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.312 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:132 3.2s0 3:47d6be26:::smithi09216568-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:38.312 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.674919+0000", 2020-11-16T20:24:38.312 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.519037309000002, 2020-11-16T20:24:38.313 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036684337999999997, 2020-11-16T20:24:38.313 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.313 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "reached pg", 2020-11-16T20:24:38.313 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.313 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.313 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.313 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 132 2020-11-16T20:24:38.314 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.314 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.314 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.314 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.314 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674919+0000", 2020-11-16T20:24:38.314 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.314 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.315 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.315 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.315 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674919+0000", 2020-11-16T20:24:38.315 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.0027e-05 2020-11-16T20:24:38.315 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.315 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.315 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.316 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674939+0000", 2020-11-16T20:24:38.316 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.6299999999999997e-06 2020-11-16T20:24:38.316 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.316 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.316 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.316 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674944+0000", 2020-11-16T20:24:38.316 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.8390000000000001e-06 2020-11-16T20:24:38.317 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.317 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.317 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.317 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674951+0000", 2020-11-16T20:24:38.317 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.7250000000000002e-06 2020-11-16T20:24:38.317 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.317 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.318 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.318 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.674953+0000", 2020-11-16T20:24:38.318 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012060743000000001 2020-11-16T20:24:38.318 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.318 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.318 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.318 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.687014+0000", 2020-11-16T20:24:38.319 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.0745000000000002e-05 2020-11-16T20:24:38.319 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.319 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.319 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.319 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.687075+0000", 2020-11-16T20:24:38.319 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.013032431000000001 2020-11-16T20:24:38.319 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.319 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.320 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.320 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700107+0000", 2020-11-16T20:24:38.320 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00011721 2020-11-16T20:24:38.320 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.320 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.320 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.320 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.700225+0000", 2020-11-16T20:24:38.321 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.011367998000000001 2020-11-16T20:24:38.321 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.321 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.321 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.321 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.711593+0000", 2020-11-16T20:24:38.321 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.099e-05 2020-11-16T20:24:38.321 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.322 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.322 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.322 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.711604+0000", 2020-11-16T20:24:38.322 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.036684337999999997 2020-11-16T20:24:38.322 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.322 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.322 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.323 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.323 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.323 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:152 3.7s0 3:eba7105b:::smithi09216568-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e31)", 2020-11-16T20:24:38.323 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.719485+0000", 2020-11-16T20:24:38.323 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 22.474471502, 2020-11-16T20:24:38.323 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.38311783900000002, 2020-11-16T20:24:38.323 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.324 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:38.324 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.324 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.324 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.324 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 152 2020-11-16T20:24:38.324 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.324 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.325 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.325 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.325 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719485+0000", 2020-11-16T20:24:38.325 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.325 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.325 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.325 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.326 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719485+0000", 2020-11-16T20:24:38.326 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.2246999999999999e-05 2020-11-16T20:24:38.326 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.326 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.326 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.326 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719507+0000", 2020-11-16T20:24:38.326 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.5299999999999998e-06 2020-11-16T20:24:38.327 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.327 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.327 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.327 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719512+0000", 2020-11-16T20:24:38.327 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.0484e-05 2020-11-16T20:24:38.327 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.327 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.328 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.328 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719522+0000", 2020-11-16T20:24:38.328 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.783e-06 2020-11-16T20:24:38.328 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.328 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.328 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.328 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.719526+0000", 2020-11-16T20:24:38.329 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.007154166 2020-11-16T20:24:38.329 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.329 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.329 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.329 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.726680+0000", 2020-11-16T20:24:38.329 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.2163999999999997e-05 2020-11-16T20:24:38.329 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.330 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.330 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.330 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.726732+0000", 2020-11-16T20:24:38.330 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012409737000000001 2020-11-16T20:24:38.330 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.330 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.330 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.331 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.739142+0000", 2020-11-16T20:24:38.331 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000102487 2020-11-16T20:24:38.331 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.331 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.331 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.331 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.739245+0000", 2020-11-16T20:24:38.331 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0022202709999999998 2020-11-16T20:24:38.332 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.332 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.332 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.332 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.741465+0000", 2020-11-16T20:24:38.332 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.8485000000000005e-05 2020-11-16T20:24:38.332 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.332 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.333 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.333 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.741543+0000", 2020-11-16T20:24:38.333 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.36105948500000001 2020-11-16T20:24:38.333 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.333 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.333 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.333 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:16.102603+0000", 2020-11-16T20:24:38.334 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.38311783900000002 2020-11-16T20:24:38.334 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.334 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.334 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.339 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.339 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.339 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:293 3.7s0 3:ea3bbe80:::smithi09216568-24:head [append 0~401408] snapc 9=[] ondisk+write+known_if_redirected e41)", 2020-11-16T20:24:38.340 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:25.777560+0000", 2020-11-16T20:24:38.340 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 12.416396552, 2020-11-16T20:24:38.340 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.031032255000000002, 2020-11-16T20:24:38.340 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.340 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.340 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.340 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.341 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.341 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 293 2020-11-16T20:24:38.341 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.341 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.341 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.341 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.341 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.777560+0000", 2020-11-16T20:24:38.342 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.342 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.342 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.342 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.342 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.777560+0000", 2020-11-16T20:24:38.342 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0010964970000000001 2020-11-16T20:24:38.342 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.343 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.343 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.343 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778656+0000", 2020-11-16T20:24:38.343 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4422e-05 2020-11-16T20:24:38.343 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.343 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.343 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.344 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778671+0000", 2020-11-16T20:24:38.344 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.5148e-05 2020-11-16T20:24:38.344 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.344 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.344 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.344 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778696+0000", 2020-11-16T20:24:38.344 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.1206000000000001e-05 2020-11-16T20:24:38.345 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.345 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.345 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.345 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.778707+0000", 2020-11-16T20:24:38.345 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0088336160000000007 2020-11-16T20:24:38.345 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.345 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.346 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.346 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.787541+0000", 2020-11-16T20:24:38.346 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00013570699999999999 2020-11-16T20:24:38.346 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.346 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.346 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.346 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.787677+0000", 2020-11-16T20:24:38.347 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00081066199999999997 2020-11-16T20:24:38.347 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.347 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.347 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.347 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.788487+0000", 2020-11-16T20:24:38.347 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0079055070000000009 2020-11-16T20:24:38.347 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.348 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.348 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.348 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.796393+0000", 2020-11-16T20:24:38.348 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.012145243 2020-11-16T20:24:38.348 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.348 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.348 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.349 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.808538+0000", 2020-11-16T20:24:38.349 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.4246999999999997e-05 2020-11-16T20:24:38.349 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.349 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.349 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.349 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:25.808592+0000", 2020-11-16T20:24:38.349 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.031032255000000002 2020-11-16T20:24:38.350 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.350 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.350 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.350 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.350 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.350 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:373 3.7s0 3:e5c1f838:::smithi09216568-15:head [append 0~614400] snapc f=[f,e,d,c] ondisk+write+known_if_redirected e45)", 2020-11-16T20:24:38.350 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:29.763528+0000", 2020-11-16T20:24:38.351 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 8.4304284169999999, 2020-11-16T20:24:38.351 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.037732144000000002, 2020-11-16T20:24:38.351 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.351 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.351 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.351 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.351 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.352 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 373 2020-11-16T20:24:38.352 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.352 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.352 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.352 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.352 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.763528+0000", 2020-11-16T20:24:38.352 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.352 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.353 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.353 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.353 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.763528+0000", 2020-11-16T20:24:38.353 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0011685580000000001 2020-11-16T20:24:38.353 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.353 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.353 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.354 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.764697+0000", 2020-11-16T20:24:38.354 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.9697000000000001e-05 2020-11-16T20:24:38.354 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.354 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.354 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.354 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.764716+0000", 2020-11-16T20:24:38.355 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.1529000000000003e-05 2020-11-16T20:24:38.355 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.355 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.355 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.355 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.764748+0000", 2020-11-16T20:24:38.355 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.5948e-05 2020-11-16T20:24:38.355 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.356 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.356 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.356 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.764764+0000", 2020-11-16T20:24:38.356 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000134843 2020-11-16T20:24:38.356 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.356 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.356 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.357 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.764899+0000", 2020-11-16T20:24:38.357 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00021225000000000001 2020-11-16T20:24:38.357 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.357 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.357 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.357 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.765111+0000", 2020-11-16T20:24:38.357 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0012141039999999999 2020-11-16T20:24:38.357 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.358 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.358 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.358 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.766325+0000", 2020-11-16T20:24:38.358 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.034667970999999999 2020-11-16T20:24:38.358 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.358 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.359 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.359 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.800993+0000", 2020-11-16T20:24:38.359 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000166486 2020-11-16T20:24:38.359 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.359 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.359 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.359 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.801159+0000", 2020-11-16T20:24:38.359 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000100758 2020-11-16T20:24:38.360 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.360 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.360 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.360 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.801260+0000", 2020-11-16T20:24:38.360 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.037732144000000002 2020-11-16T20:24:38.360 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.360 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.361 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.361 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.361 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.361 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:374 3.7s0 3:e5c1f838:::smithi09216568-15:head [append 0~434176] snapc f=[f,e,d,c] ondisk+write+known_if_redirected e45)", 2020-11-16T20:24:38.361 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:29.769856+0000", 2020-11-16T20:24:38.361 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 8.4241007260000007, 2020-11-16T20:24:38.361 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.031710734999999997, 2020-11-16T20:24:38.362 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.362 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.362 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.362 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.362 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.362 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 374 2020-11-16T20:24:38.362 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.363 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.363 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.363 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.363 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.769856+0000", 2020-11-16T20:24:38.363 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.363 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.363 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.364 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.364 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.769856+0000", 2020-11-16T20:24:38.364 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00062642100000000005 2020-11-16T20:24:38.364 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.364 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.364 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.364 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.770482+0000", 2020-11-16T20:24:38.365 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4528e-05 2020-11-16T20:24:38.365 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.365 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.365 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.365 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.770497+0000", 2020-11-16T20:24:38.365 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.4902999999999999e-05 2020-11-16T20:24:38.365 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.366 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.366 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.366 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.770522+0000", 2020-11-16T20:24:38.366 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.2140000000000001e-05 2020-11-16T20:24:38.366 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.366 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.366 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.367 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.770534+0000", 2020-11-16T20:24:38.367 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0076798550000000002 2020-11-16T20:24:38.367 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.367 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.367 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.367 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.778214+0000", 2020-11-16T20:24:38.367 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00027934299999999998 2020-11-16T20:24:38.368 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.368 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.368 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.368 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.778493+0000", 2020-11-16T20:24:38.368 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0014359819999999999 2020-11-16T20:24:38.368 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.368 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.369 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.369 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.779929+0000", 2020-11-16T20:24:38.369 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.021066054000000001 2020-11-16T20:24:38.369 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.369 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.369 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.369 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.800995+0000", 2020-11-16T20:24:38.370 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00046766700000000001 2020-11-16T20:24:38.370 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.370 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.370 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.370 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.801463+0000", 2020-11-16T20:24:38.370 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000103842 2020-11-16T20:24:38.370 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.371 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.371 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.371 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.801566+0000", 2020-11-16T20:24:38.371 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.031710734999999997 2020-11-16T20:24:38.371 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.371 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.371 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.372 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.372 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.372 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:375 3.7s0 3:e5c1f838:::smithi09216568-15:head [append 0~368640] snapc f=[f,e,d,c] ondisk+write+known_if_redirected e45)", 2020-11-16T20:24:38.372 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:29.772707+0000", 2020-11-16T20:24:38.372 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 8.4212491420000006, 2020-11-16T20:24:38.372 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030671555, 2020-11-16T20:24:38.372 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.373 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.373 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.373 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.373 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.373 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 375 2020-11-16T20:24:38.373 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.373 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.374 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.374 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.374 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.772707+0000", 2020-11-16T20:24:38.374 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.374 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.374 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.374 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.375 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.772707+0000", 2020-11-16T20:24:38.375 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000648136 2020-11-16T20:24:38.375 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.375 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.375 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.375 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773355+0000", 2020-11-16T20:24:38.375 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.5528999999999999e-05 2020-11-16T20:24:38.375 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.376 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.376 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.376 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773381+0000", 2020-11-16T20:24:38.376 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.3111e-05 2020-11-16T20:24:38.376 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.376 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.376 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.377 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773434+0000", 2020-11-16T20:24:38.377 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.6503000000000001e-05 2020-11-16T20:24:38.377 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.377 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.377 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.377 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773451+0000", 2020-11-16T20:24:38.377 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.017137716000000001 2020-11-16T20:24:38.378 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.378 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.378 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.378 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.790588+0000", 2020-11-16T20:24:38.378 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00033812300000000002 2020-11-16T20:24:38.378 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.378 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.379 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.379 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.790926+0000", 2020-11-16T20:24:38.379 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0016186250000000001 2020-11-16T20:24:38.379 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.379 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.379 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.379 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.792545+0000", 2020-11-16T20:24:38.380 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.01061615 2020-11-16T20:24:38.380 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.380 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.380 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.380 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.803161+0000", 2020-11-16T20:24:38.380 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000114344 2020-11-16T20:24:38.380 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.381 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.381 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.381 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.803276+0000", 2020-11-16T20:24:38.381 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000103318 2020-11-16T20:24:38.381 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.381 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.381 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.382 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.803379+0000", 2020-11-16T20:24:38.382 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030671555 2020-11-16T20:24:38.382 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.382 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.382 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.382 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.382 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.383 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:376 3.7s0 3:e5c1f838:::smithi09216568-15:head [setxattr _header (56) in=7b] snapc f=[f,e,d,c] ondisk+write+known_if_redirected e45)", 2020-11-16T20:24:38.383 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:29.773567+0000", 2020-11-16T20:24:38.383 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 8.4203898430000006, 2020-11-16T20:24:38.383 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030367811000000001, 2020-11-16T20:24:38.383 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.383 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.383 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.384 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.384 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.384 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 376 2020-11-16T20:24:38.384 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.384 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.384 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.384 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.385 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773567+0000", 2020-11-16T20:24:38.385 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.385 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.385 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.385 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.385 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773567+0000", 2020-11-16T20:24:38.385 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000143232 2020-11-16T20:24:38.385 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.386 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.386 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.386 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773710+0000", 2020-11-16T20:24:38.386 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.732e-05 2020-11-16T20:24:38.386 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.386 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.386 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.387 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773727+0000", 2020-11-16T20:24:38.387 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.7966000000000002e-05 2020-11-16T20:24:38.387 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.387 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.387 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.387 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773765+0000", 2020-11-16T20:24:38.387 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.1911000000000001e-05 2020-11-16T20:24:38.388 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.388 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.388 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.388 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773787+0000", 2020-11-16T20:24:38.388 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.027951554 2020-11-16T20:24:38.388 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.388 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.389 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.389 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.801739+0000", 2020-11-16T20:24:38.389 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.6045999999999995e-05 2020-11-16T20:24:38.389 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.389 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.389 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.389 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.801835+0000", 2020-11-16T20:24:38.390 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000419503 2020-11-16T20:24:38.390 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.390 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.390 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.390 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.802254+0000", 2020-11-16T20:24:38.390 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0011463879999999999 2020-11-16T20:24:38.390 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.391 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.391 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.391 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.803401+0000", 2020-11-16T20:24:38.391 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00047193400000000003 2020-11-16T20:24:38.391 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.391 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.391 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.392 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.803873+0000", 2020-11-16T20:24:38.392 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.1957000000000003e-05 2020-11-16T20:24:38.392 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.392 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.392 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.392 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.803934+0000", 2020-11-16T20:24:38.392 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.030367811000000001 2020-11-16T20:24:38.393 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.393 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.393 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.393 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.393 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.393 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:377 3.7s0 3:e5c1f838:::smithi09216568-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2020-11-16T20:24:38.393 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:29.773890+0000", 2020-11-16T20:24:38.394 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 8.4200663799999997, 2020-11-16T20:24:38.394 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.031712775999999998, 2020-11-16T20:24:38.394 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.394 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:38.394 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.394 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.394 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.395 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 377 2020-11-16T20:24:38.395 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.395 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.395 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.395 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.395 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773890+0000", 2020-11-16T20:24:38.395 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.396 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.396 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.396 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.396 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773890+0000", 2020-11-16T20:24:38.396 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000102872 2020-11-16T20:24:38.396 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.396 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.397 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.397 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.773993+0000", 2020-11-16T20:24:38.397 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.6708999999999999e-05 2020-11-16T20:24:38.397 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.397 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.397 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.397 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.774010+0000", 2020-11-16T20:24:38.398 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.7925e-05 2020-11-16T20:24:38.398 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.398 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.398 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.398 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.774048+0000", 2020-11-16T20:24:38.398 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.1977000000000001e-05 2020-11-16T20:24:38.398 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.399 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.399 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.399 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.774060+0000", 2020-11-16T20:24:38.399 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.028576900999999998 2020-11-16T20:24:38.399 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.399 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.399 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.400 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.802636+0000", 2020-11-16T20:24:38.400 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00012655300000000001 2020-11-16T20:24:38.400 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.400 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.400 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.400 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.802763+0000", 2020-11-16T20:24:38.400 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00148113 2020-11-16T20:24:38.401 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.401 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.401 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.401 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.804244+0000", 2020-11-16T20:24:38.401 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00010442699999999999 2020-11-16T20:24:38.401 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.401 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.402 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.402 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.804349+0000", 2020-11-16T20:24:38.402 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.001254282 2020-11-16T20:24:38.402 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.402 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.402 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.402 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:29.805603+0000", 2020-11-16T20:24:38.403 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.031712775999999998 2020-11-16T20:24:38.403 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.403 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.403 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.403 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.403 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.403 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:414 3.7s0 3:e2218301:::smithi09216568-21:head [copy-from ver 26] snapc 10=[10,f,e,c] ondisk+write+known_if_redirected e47)", 2020-11-16T20:24:38.404 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:31.600320+0000", 2020-11-16T20:24:38.404 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 6.5936363870000001, 2020-11-16T20:24:38.404 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.032146902999999998, 2020-11-16T20:24:38.404 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.404 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "commit sent; apply or cleanup", 2020-11-16T20:24:38.404 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.404 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.405 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.405 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 414 2020-11-16T20:24:38.405 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.405 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.405 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.405 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.405 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600320+0000", 2020-11-16T20:24:38.405 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.406 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.406 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.406 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.406 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600320+0000", 2020-11-16T20:24:38.406 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 4.0889000000000001e-05 2020-11-16T20:24:38.406 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.406 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.407 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.407 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600361+0000", 2020-11-16T20:24:38.407 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.9100000000000005e-06 2020-11-16T20:24:38.407 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.407 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.407 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.407 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600369+0000", 2020-11-16T20:24:38.408 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.9332999999999999e-05 2020-11-16T20:24:38.408 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.408 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.408 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.408 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600398+0000", 2020-11-16T20:24:38.408 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.1999999999999997e-06 2020-11-16T20:24:38.408 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.409 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.409 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.409 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600405+0000", 2020-11-16T20:24:38.409 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.7542999999999999e-05 2020-11-16T20:24:38.409 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.409 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.409 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.410 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600483+0000", 2020-11-16T20:24:38.410 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00018089899999999999 2020-11-16T20:24:38.410 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.410 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.410 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.410 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600664+0000", 2020-11-16T20:24:38.410 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.01162816 2020-11-16T20:24:38.411 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.411 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.411 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.411 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.612292+0000", 2020-11-16T20:24:38.411 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.014161236000000001 2020-11-16T20:24:38.411 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.411 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.412 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.412 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.626453+0000", 2020-11-16T20:24:38.412 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.005872492 2020-11-16T20:24:38.412 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.412 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.412 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "commit_sent", 2020-11-16T20:24:38.412 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.632326+0000", 2020-11-16T20:24:38.413 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00014124100000000001 2020-11-16T20:24:38.413 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.413 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.413 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.413 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.632467+0000", 2020-11-16T20:24:38.413 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.032146902999999998 2020-11-16T20:24:38.413 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.414 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.414 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.414 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.414 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.414 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "osd_op(client.4444.0:415 3.7s0 3:e2218301:::smithi09216568-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2020-11-16T20:24:38.414 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:31.600507+0000", 2020-11-16T20:24:38.414 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 6.593449412, 2020-11-16T20:24:38.415 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.032745346000000002, 2020-11-16T20:24:38.415 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.415 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "started", 2020-11-16T20:24:38.415 INFO:teuthology.orchestra.run.smithi013.stdout: "client_info": { 2020-11-16T20:24:38.415 INFO:teuthology.orchestra.run.smithi013.stdout: "client": "client.4444", 2020-11-16T20:24:38.415 INFO:teuthology.orchestra.run.smithi013.stdout: "client_addr": "172.21.15.92:0/3479785109", 2020-11-16T20:24:38.415 INFO:teuthology.orchestra.run.smithi013.stdout: "tid": 415 2020-11-16T20:24:38.416 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.416 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.416 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.416 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.416 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600507+0000", 2020-11-16T20:24:38.416 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.416 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.416 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.417 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.417 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600507+0000", 2020-11-16T20:24:38.417 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 5.6419999999999999e-05 2020-11-16T20:24:38.417 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.417 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.417 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.417 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600563+0000", 2020-11-16T20:24:38.418 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.1671000000000002e-05 2020-11-16T20:24:38.418 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.418 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.418 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.418 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600595+0000", 2020-11-16T20:24:38.418 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.8711999999999999e-05 2020-11-16T20:24:38.419 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.419 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.419 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.419 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600624+0000", 2020-11-16T20:24:38.419 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.046e-05 2020-11-16T20:24:38.419 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.419 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.419 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.420 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600634+0000", 2020-11-16T20:24:38.420 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00027939000000000002 2020-11-16T20:24:38.420 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.420 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.420 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.420 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600914+0000", 2020-11-16T20:24:38.420 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.9726000000000002e-05 2020-11-16T20:24:38.421 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.421 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.421 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for blocked object", 2020-11-16T20:24:38.421 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.600993+0000", 2020-11-16T20:24:38.421 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.024399460000000001 2020-11-16T20:24:38.421 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.421 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.422 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.422 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.625393+0000", 2020-11-16T20:24:38.422 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00014259799999999999 2020-11-16T20:24:38.422 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.422 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.422 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "waiting for rw locks", 2020-11-16T20:24:38.422 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.625536+0000", 2020-11-16T20:24:38.423 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0075485489999999999 2020-11-16T20:24:38.423 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.423 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.423 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.423 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.633084+0000", 2020-11-16T20:24:38.423 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 7.9567000000000006e-05 2020-11-16T20:24:38.423 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.424 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.424 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "started", 2020-11-16T20:24:38.424 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.633164+0000", 2020-11-16T20:24:38.424 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.8793000000000002e-05 2020-11-16T20:24:38.424 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.424 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.424 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.425 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:31.633252+0000", 2020-11-16T20:24:38.425 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.032745346000000002 2020-11-16T20:24:38.425 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.425 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.425 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.425 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.425 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.426 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:38.427 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-11-16T20:24:38.428 INFO:tasks.ceph.osd.6.smithi013.stderr:2020-11-16T20:24:38.428+0000 7f7391cef700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 12988) UID: 0 2020-11-16T20:24:38.528 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:38.524+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:38.557 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:38.557 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:38.557 INFO:teuthology.orchestra.run.smithi013.stdout: "num_ops": 0 2020-11-16T20:24:38.557 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:38.568 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-11-16T20:24:38.579 INFO:tasks.rados.rados.0.smithi092.stdout:196: done (2 left) 2020-11-16T20:24:38.579 INFO:tasks.rados.rados.0.smithi092.stdout:append oid 32 current snap is 14 2020-11-16T20:24:38.580 INFO:tasks.rados.rados.0.smithi092.stdout:197: seq_num 99 ranges {1925120=466944,2392064=802816,3194880=401408} 2020-11-16T20:24:38.594 INFO:tasks.rados.rados.0.smithi092.stdout:197: writing smithi09216568-32 from 1925120 to 2392064 tid 1 2020-11-16T20:24:38.600 INFO:tasks.rados.rados.0.smithi092.stdout:197: writing smithi09216568-32 from 2392064 to 3194880 tid 2 2020-11-16T20:24:38.603 INFO:tasks.rados.rados.0.smithi092.stdout:197: writing smithi09216568-32 from 3194880 to 3596288 tid 3 2020-11-16T20:24:38.603 INFO:tasks.rados.rados.0.smithi092.stdout:198: snap_remove snap 7 2020-11-16T20:24:38.660 INFO:tasks.ceph.osd.5.smithi013.stderr:2020-11-16T20:24:38.660+0000 7feae9455700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 12984) UID: 0 2020-11-16T20:24:38.729 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:38.729 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [], 2020-11-16T20:24:38.729 INFO:teuthology.orchestra.run.smithi013.stdout: "complaint_time": 30, 2020-11-16T20:24:38.729 INFO:teuthology.orchestra.run.smithi013.stdout: "num_blocked_ops": 0 2020-11-16T20:24:38.729 INFO:teuthology.orchestra.run.smithi013.stdout:} 2020-11-16T20:24:38.730 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:38.729+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:38.739 INFO:teuthology.orchestra.run.smithi013:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-11-16T20:24:38.830 INFO:tasks.ceph.osd.2.smithi092.stderr:2020-11-16T20:24:38.829+0000 7f1e654cd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 12728) UID: 0 2020-11-16T20:24:38.902 INFO:teuthology.orchestra.run.smithi013.stdout:{ 2020-11-16T20:24:38.903 INFO:teuthology.orchestra.run.smithi013.stdout: "size": 20, 2020-11-16T20:24:38.903 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 600, 2020-11-16T20:24:38.903 INFO:teuthology.orchestra.run.smithi013.stdout: "ops": [ 2020-11-16T20:24:38.903 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.903 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "MOSDECSubOpWrite(3.4s1 31/23 ECSubWrite(tid=0, reqid=client.4444.0:4, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:38.903 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.532286+0000", 2020-11-16T20:24:38.904 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 23.37113343, 2020-11-16T20:24:38.904 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.011231474, 2020-11-16T20:24:38.904 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.904 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "reached pg", 2020-11-16T20:24:38.904 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.904 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.904 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.905 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.532286+0000", 2020-11-16T20:24:38.905 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.905 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.905 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.905 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.905 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.532286+0000", 2020-11-16T20:24:38.905 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.4759999999999995e-05 2020-11-16T20:24:38.906 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.906 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.906 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.906 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.532371+0000", 2020-11-16T20:24:38.906 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 3.9830000000000003e-05 2020-11-16T20:24:38.906 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.906 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.906 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.907 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.532411+0000", 2020-11-16T20:24:38.907 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.00014814 2020-11-16T20:24:38.907 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.907 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.907 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.907 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.532559+0000", 2020-11-16T20:24:38.907 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.4829e-05 2020-11-16T20:24:38.908 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.908 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.908 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.908 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.532574+0000", 2020-11-16T20:24:38.908 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.000142843 2020-11-16T20:24:38.908 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.908 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.908 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.909 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.532716+0000", 2020-11-16T20:24:38.909 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.9426999999999999e-05 2020-11-16T20:24:38.909 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.909 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.909 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.909 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.532736+0000", 2020-11-16T20:24:38.909 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.010715538 2020-11-16T20:24:38.910 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.910 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.910 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.910 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.543451+0000", 2020-11-16T20:24:38.910 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.6106999999999996e-05 2020-11-16T20:24:38.910 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.910 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.911 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.911 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.543518+0000", 2020-11-16T20:24:38.911 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.011231474 2020-11-16T20:24:38.911 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.911 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.911 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.911 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.911 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.912 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "MOSDECSubOpWrite(3.4s1 31/23 ECSubWrite(tid=2, reqid=client.4444.0:19, at_version=31'2, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:38.912 INFO:teuthology.orchestra.run.smithi013.stdout: "initiated_at": "2020-11-16T20:24:15.540418+0000", 2020-11-16T20:24:38.912 INFO:teuthology.orchestra.run.smithi013.stdout: "age": 23.363001314000002, 2020-11-16T20:24:38.912 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0057316169999999996, 2020-11-16T20:24:38.912 INFO:teuthology.orchestra.run.smithi013.stdout: "type_data": { 2020-11-16T20:24:38.912 INFO:teuthology.orchestra.run.smithi013.stdout: "flag_point": "reached pg", 2020-11-16T20:24:38.912 INFO:teuthology.orchestra.run.smithi013.stdout: "events": [ 2020-11-16T20:24:38.912 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.913 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "initiated", 2020-11-16T20:24:38.913 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.540418+0000", 2020-11-16T20:24:38.913 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0 2020-11-16T20:24:38.913 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.913 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.913 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "throttled", 2020-11-16T20:24:38.913 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.540418+0000", 2020-11-16T20:24:38.914 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 8.8399999999999994e-05 2020-11-16T20:24:38.914 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.914 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.914 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "header_read", 2020-11-16T20:24:38.914 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.540507+0000", 2020-11-16T20:24:38.914 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 2.7617999999999999e-05 2020-11-16T20:24:38.914 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.914 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.915 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "all_read", 2020-11-16T20:24:38.915 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.540534+0000", 2020-11-16T20:24:38.915 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.7938999999999999e-05 2020-11-16T20:24:38.915 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.915 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.915 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "dispatched", 2020-11-16T20:24:38.915 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.540632+0000", 2020-11-16T20:24:38.916 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.0243e-05 2020-11-16T20:24:38.916 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.916 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.916 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "queued_for_pg", 2020-11-16T20:24:38.916 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.540642+0000", 2020-11-16T20:24:38.916 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 9.0957000000000003e-05 2020-11-16T20:24:38.916 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.916 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.917 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "reached_pg", 2020-11-16T20:24:38.917 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.540733+0000", 2020-11-16T20:24:38.917 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 1.8111e-05 2020-11-16T20:24:38.917 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.917 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.917 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_started", 2020-11-16T20:24:38.917 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.540751+0000", 2020-11-16T20:24:38.918 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0053287100000000004 2020-11-16T20:24:38.918 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.918 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.918 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "sub_op_committed", 2020-11-16T20:24:38.918 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.546080+0000", 2020-11-16T20:24:38.918 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 6.9639000000000002e-05 2020-11-16T20:24:38.918 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.918 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.919 INFO:teuthology.orchestra.run.smithi013.stdout: "event": "done", 2020-11-16T20:24:38.919 INFO:teuthology.orchestra.run.smithi013.stdout: "time": "2020-11-16T20:24:15.546150+0000", 2020-11-16T20:24:38.919 INFO:teuthology.orchestra.run.smithi013.stdout: "duration": 0.0057316169999999996 2020-11-16T20:24:38.919 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.919 INFO:teuthology.orchestra.run.smithi013.stdout: ] 2020-11-16T20:24:38.919 INFO:teuthology.orchestra.run.smithi013.stdout: } 2020-11-16T20:24:38.919 INFO:teuthology.orchestra.run.smithi013.stdout: }, 2020-11-16T20:24:38.920 INFO:teuthology.orchestra.run.smithi013.stdout: { 2020-11-16T20:24:38.920 INFO:teuthology.orchestra.run.smithi013.stdout: "description": "MOSDECSubOpWrite(3.es2 31/23 ECSubWrite(tid=1, reqid=client.4444.0:58, at_version=31'1, trim_to=0'0, roll_forward_to=0'0))", 2020-11-16T20:24:38.920 INFO:teuthology.or